Mail Deliverability Test

Frequents breaks in the availability of Exchange Online and prolonged slowness in delivery of mails sent/received via Exchange Online, can adversely impact user experience with Exchange Online. To assure users of a high quality experience with Exchange Online at all times, administrators should be able to proactively detect and promptly avert the non-availability of Exchange Online and any processing slowness that it may be experiencing. This is where the Mail Deliverability test helps!

At configured intervals, this test emulates a user sending/receiving a configured number of emails (default: 1) over Exchange Online. In the process, the test verifies the availability of Exchange Online for sending and receiving the emulated mail(s), and also reports the time taken to send and receive mails. This way, the test notifies administrators of the non-availability of Exchange Online and processing bottlenecks that it may be experiencing, well before users notice and complain.

Target of the test : Exchange Online

Agent deploying the test : A remote agent

Outputs of the test : One set of results for the monitored Office 365 tenant

Configurable parameters for the test

Parameters Description

Test period

How often should the test be executed

Host

The host for which the test is to be configured. By default, this is portal.office.com

Tenant Name

This parameter applies only if you want the eG agent to use Azure AD Certificate-based Authentication for accessing and monitoring an O365 tenant and its resources.

Azure AD certificate-based authentication (CBA) enables customers to allow or require users to authenticate with X.509 certificates against their Azure Active Directory (Azure AD) for applications and browser sign-in. When monitoring highly secure Office 365 environments, you can configure the eG agent to identify itself to a tenant using a valid X.509 certificate, so that it is allowed secure access to the tenant and its resources.

By default, the value of this parameter is none. This means that, by default, the eG agent does not use certificate-based authentication to connect to an O365 tenant.

On the other hand, if you want the eG agent to use this modern authentication technique to securely access a tenant's resources, you should do the following:

  1. Enable Azure AD Certificate-based authentication for the target O365 tenant; this can be achieved manually, via the Office 365 portal, or automatically, using Powershell scripts we provide. For the manual procedure, refer to Manually Enabling Certificate-based Authentication For an Office 365 Tenantunder Microsoft Office 365. For the automatic procedure, refer to Automatically Fulfilling Pre-requisites in a Modern Authentication-Enabled Environmentunder Microsoft Office 365.

    When enabling certificate-based authentication, an X.509 certificate will be generated for the target tenant.

  2. Configure the Tenant Name parameter with the name of the tenant for which certificate-based authentication is enabled. Using the tenant name, the eG agent will be able to read the details of the X.509 certificate that is generated for that tenant, and use that certificate to access that tenant's resources. To determine the tenant name, do the following:

    • Log in to the Microsoft 365 Admin Center as an administrator.

    • Under Setup, click on Domains.

    • Find a domain that ends with .onmicrosoft.com - this is your Microsoft O365 tenant name.

O365 User Name, O365 Password, and Confirm Password

These parameters need to be configured only if the Tenant Name parameter is set to none. On the other hand, if a valid Tenant Name is configured, then you should set these parameters to none .

For execution, this test requires the privileges of an O365 user who is vested with the View-Only Audit Logs, View-Only Recipients, Mail Recipients, and Mailbox Import Export permissions. Configure the credentials of such a user against O365 User Name and O365 Password text boxes. Confirm the password by retyping it in the Confirm Password text box.

While you can use the credentials of any existing O365 user with the afore-said privileges, it is recommended that you create a special user for monitoring purposes using the Office 365 portal and use the credentials of that user here. To know how to create a new user using the Office 365 portal and assign the required privileges to that user, refer to Creating a New User in the Office 365 Portal under Microsoft Office 365. You can also use eG's proprietary PowerShell script to automatically create a new user, or assign the required privileges to an existing user. To know how to use this script, refer to theAutomatically Fulfilling Pre-requisites in a Basic Authentication-Enabled Environmenttopic.

Domain, Domain User Name, Domain Password, and Confirm Password

These parameters are applicable only if the eG agent needs to communicate with the Office 365 portal via a Proxy server.

In this case, in the Domain text box, specify the name of the Windows domain to which the eG agent host belongs. In the Domain User Name text box, mention the name of a valid domain user with login rights to the eG agent host. Provide the password of that user in the Domain Password text box and confirm that password by retyping it in the Confirm Password text box.

On the other hand, if the eG agent is not behind a Proxy server, then you need not disturb the default setting of these parameters. By default, these parameters are set to none.

Proxy Host, Proxy Port, Proxy User Name, and Proxy Password

These parameters are applicable only if the eG agent needs to communicate with the Office 365 portal via a Proxy server.

In this case, provide the IP/host name and port number of the Proxy server that the eG agent should use in the Proxy Host and Proxy Port parameters, respectively.

If the Proxy server requires authentication, then specify the credentials of a valid Proxy user against the Proxy User Name and Proxy Password text boxes. Confirm that password by retyping it in the Confirm Password text box. If the Proxy server does not require authentication, then specify none against the Proxy User Name, Proxy Password, and Confirm Password text boxes.

On the other hand, if the eG agent is not behind a Proxy server, then you need not disturb the default setting of any of the Proxy-related parameters. By default, these parameters are set to none.

Number of Messages

Specify the number of messages this test should send over Exchange Online, as part of the emulation. By default, this parameter is set to 1.

SMTP Port

This test uses SMTP protocol to send mails over Exchange online. To enable the test to use SMTP, administrators need to specify the SMTP port number against SMTP Port text box. By default, the port number is set to 587. However, if the SMTP port is different from the default value, then specify that value here.

Use SSL

By default, the Use SSL flag is set to Yes indicating that the test sends mails through secure connection. However, you can set this flag to No if you do not want this test to transfer mails through secure connection.

Measurements made by the test
Measurement Description Measurement Unit Interpretation

Send mail availability

Indicates whether/not Exchange Online is available to send emails.

Percent

If the value of this measure is 100%, it implies that Exchange Online is available for sending emails. The value 0 on the other hand denotes that Exchange Online is not available for sending emails.

Sent messages

Indicates the number of messages this test sent successfully over Exchange Online.

Number

If this test has been configured to send more than one email over Exchange Online (via the Number of Messages parameter), then the value of this measure will clearly indicate whether all messages were successfully sent or not, and if not, how many email transmissions failed.

Avg time to send messages

Indicates the average time taken by Exchange Online to send messages.

Seconds

Ideally, the value of this measure should be low. A high value is indicative of a bottleneck when sending messages.

Receive mail availability

Indicates whether/not Exchange Online is available to receive emails.

Percent

If the value of this measure is 100%, it implies that Exchange Online is available for receiving emails. The value 0 on the other hand denotes that Exchange Online is not available for receiving emails.

Received messages

Indicates the number of messages this test received successfully over Exchange Online.

Number

If say, this test has been configured to receive more than one email over Exchange Online (via the Number of Messages parameter), then the value of this measure will clearly indicate whether all messages were successfully received or not, and if not, how many email emails could not be received.

Avg time to receive messages

Indicates the average time taken by Exchange Online to receive messages.

Seconds

Ideally, the value of this measure should be low. A high value is indicative of a bottleneck when receiving messages.

Avg round-trip time

Indicates the average time taken to send a message over Exchange Online and receive it.

Minutes

Ideally, the value of this measure should be low. A high value is indicative of a bottleneck when receiving messages.

Max round-trip time

Indicates the maximum time taken to send a message over Exchange Online and receive it.

Minutes