Azure Event Hubs Test

Azure Event Hubs is a big data streaming platform and event ingestion service. It can receive and process millions of events per second.

Typically, Event Hubs represent the "front door" for an event pipeline, often called an event ingestor in solution architectures. An event ingestor is a component or service that sits between event publishers(a.k.a event producers) and event consumers (a.k.a event receivers) to decouple the production of an event stream from the consumption of those events.

Data sent to an event hub can also be transformed and stored using any real-time analytics provider or batching/storage adapters.

The advanced analytics that can be derived from the event hubs play a crucial role in anomaly detection, application logging, transaction processing, data archiving, and more. In such scenarios naturally, errors in the event hub or the failure of the hub to respond to requests from clients/consumers, can result in serious consequences - eg., delays in problem detection, lapses in application logging, etc.

To avoid this, it is important that administrators constantly monitor the status of the Event hubs, continuously measure their ability to process requests, and promptly capture and resolve request processing bottlenecks, before they disrupt the critical services that these hubs provide. This is exactly what the Azure Event Hubs test does!

This test auto-discovers the Event hubs created for the target subscription. For each Event hub, the test then reports the hub status, so that hubs in an abnormal state can be quickly identified. Additionally, the test also promptly alerts administrators to errors impacting a hub's operations, and pinpoints what could be causing those errors - a problematic client? or a faulty server? Furthermore, the test tracks requests received by each hub from clients, observes how every hub processes those requests, and raises an alarm if any hub fails to service the requests. This way, the test enables administrators to rapidly capture and fix hub-related anomalies, so that every hub provides uninterrupted event streaming services.

Target of the Test: A Microsoft Azure Subscription

Agent deploying the test: A remote agent

Output of the test: One set of results for each Event Hub configured for every resource group in the target Azure Subscription

Configurable parameters for the test
Parameters Description

Test Period

How often should the test be executed.

Host

The host for which the test is to be configured.

Subscription ID

Specify the GUID which uniquely identifies the Microsoft Azure Subscription to be monitored. To know the ID that maps to the target subscription, do the following:

  1. Login to the Microsoft Azure Portal.

  2. When the portal opens, click on the Subscriptions option (as indicated by Figure 1).

    Figure 1 : Clicking on the Subscriptions option

  3. Figure 2 that appears next will list all the subscriptions that have been configured for the target Azure AD tenant. Locate the subscription that is being monitored in the list, and check the value displayed for that subscription in the Subscription ID column.

    Figure 2 : Determining the Subscription ID

  4. Copy the Subscription ID in Figure 2 to the text box corresponding to the SUBSCRIPTION ID parameter in the test configurationpage.

Tenant ID

Specify the Directory ID of the Azure AD tenant to which the target subscription belongs. To know how to determine the Directory ID, refer to Configuring the eG Agent to Monitor a Microsoft Azure Subscription Using Azure ARM REST API.

Client ID, Client Password, and Confirm Password

To connect to the target subscription, the eG agent requires an Access token in the form of an Application ID and the client secret value. For this purpose, you should register a new application with the Azure AD tenant. To know how to create such an application and determine its Application ID and client secret, refer to Configuring the eG Agent to Monitor a Microsoft Azure Subscription Using Azure ARM REST API. Specify the Application ID of the created Application in the Client ID text box and the client secret value in the Client Password text box. Confirm the Client Password by retyping it in the Confirm Password text box.

Proxy Host and Proxy Port

In some environments, all communication with the Azure cloud be routed through a proxy server. In such environments, you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics. To enable metrics collection via a proxy, specify the IP address of the proxy server and the port at which the server listens against the Proxy Host and Proxy Port parameters. By default, these parameters are set to none, indicating that the eG agent is not configured to communicate via a proxy, by default.

Proxy Username, Proxy Password and Confirm Password

If the proxy server requires authentication, then, specify a valid proxy user name and password in the Proxy Username and Proxy Password parameters, respectively. Then, confirm the password by retyping it in the Confirm Password text box.

Detailed Diagnosis

To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

  • The eG manager license should allow the detailed diagnosis capability
  • Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.
Measures made by the test:
Measurement Description Measurement Unit Interpretation

Status

Indicates the current status of this Event hub.

 

The values reported by this measure and its numeric equivalents are mentioned in the table below:

Measure Value Numeric Value
Creating 1
Created 2
Activating 3
Enabling 4
Active 5
Disabling 6
Disabled 7
Soft Deleting 8
Soft Deleted 9
Removing 10
Removed 11
Failed 12
Unknown 0

Note:

By default, this measure reports the Measure Values listed in the table above to indicate the current status of the the Event hub. In the graph of this measure however, the same is represented using the numeric equivalents only.

Use the detailed diagnosis of this measure to know the location of the event hub, when it was created, the pricing tier, and throughput units assigned to the hub.

Provisioning status

Indicates the provisioning status of this Event hub.

 

The values reported by this measure and its numeric equivalents are mentioned in the table below:

Measure Value Numeric Value
Succeeded 1
Updating 2
Error 3
Unknown 0

Note:

By default, this measure reports the Measure Values listed in the table above to indicate the current provisioning status of the Event hub. In the graph of this measure however, the same is represented using the numeric equivalents only.

Incoming send requests

Indicates the number of requests made to this Event Hub.

Number

 

Successful requests

Indicates the number of successful requests made to this Event hub.

Number

Ideally, the value of this measure should be equal to or close to the value of the Incoming send requests measure. On the other hand, if the value of this measure is much lower than the value of the Incoming send requests measure, it implies that many requests failed.

Failed requests

Indicates the number of requests to this Event hub that failed.

Number

Ideally, the value of this measure should be 0 or very low.

Server busy errors

Indicates the number of ServerBusy errors that this Event hub encountered.

Number

A ServerBusyException is thrown when a server is busy. Callers should wait a while and retry the operation.

Ideally, the value of this measure should be 0.

Internal server errors

Indicates the number of internal server errors sent back to the sender or receiver while performing run-time operations.

Number

InternalServerErrors occur due to either service-side or network problems.

Ideally, the value of this measure should be 0.

Other errors

Indicates the number of OtherErrors.

Number

OtherErrors are types of errors are due to faults at the sender or receiver side, such as providing bad parameters, not enough credentials, or trying to perform an operation on a nonexistent entity.

Ideally, the value of this measure should be 0.

Incoming messages

Indicates the number of events or messages sent this Event hub.

Number

 

Outgoing messages

Indicates the number of events or messages received from this Event hub.

Number

 

Incoming bytes

Indicates the incoming bytes for this Event hub.

MB

 

Outgoing bytes

Indicates the outgoing bytes for this Event hub.

MB

 

Archive backlog messages

Indicates the number of archive messages in the backlog of this Event hub.

Number

 

Archive messages

Indicates the number of archive messages in this Event hub.

Number

 

Archive message throughput

Indicates the archive message throughput for the Event hub.

MB