Azure Key Vault Test

Azure Key Vault is a cloud service for securely storing and accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic keys.

Anybody with an Azure subscription can create and use key vaults. Typically, Key Vaults benefit developers, security administrators, and even an organization's adminstrator who manages other Azure services. Authorized users can create a vault, create/import a key/secret in the vault, and can also authorize users/applications to access the vault to manage or use its contents.

If any application's attempt to access a vault ends in failure, or if any application experiences slowness during vault access, then that application will not be able to use the passwords/keys/certificates stored in the vault as and when it needs it. As a result, application performance will deteriorate. To avoid this, administrators should continuously monitor the status of and service levels delivered by every vault configured for an Azure subscription, and rapidly isolate the vault experiencing errors / failures / processing bottlenecks. This is where the Azure Key Vault test helps!

This test auto-discovers the Key Vaults configured for an Azure subscription. For each vault so discovered, this test then tracks the provisioning status of that vault, and notifies administrators if errors are noticed during provisioning. The test also tracks the hits/requests to each vault, and measures the time taken by that vault to service the requests. This way, the test pinpoints the precise vault that is slow in processing requests to it. Additionally, the test also indicates the exact type of requests that a vault is having trouble processing - requests for keys? secrets? passwords? or others?. This eases troubleshooting.

Target of the Test: A Microsoft Azure Subscription

Agent deploying the test: A remote agent

Output of the test: One set of results for each Azure Key Vault configured for every resource group in the target Azure Subscription

Configurable parameters for the test
Parameters Description

Test Period

How often should the test be executed.

Host

The host for which the test is to be configured.

Subscription ID

Specify the GUID which uniquely identifies the Microsoft Azure Subscription to be monitored. To know the ID that maps to the target subscription, do the following:

  1. Login to the Microsoft Azure Portal.

  2. When the portal opens, click on the Subscriptions option (as indicated by Figure 1).

    Figure 1 : Clicking on the Subscriptions option

  3. Figure 2 that appears next will list all the subscriptions that have been configured for the target Azure AD tenant. Locate the subscription that is being monitored in the list, and check the value displayed for that subscription in the Subscription ID column.

    Figure 2 : Determining the Subscription ID

  4. Copy the Subscription ID in Figure 2 to the text box corresponding to the SUBSCRIPTION ID parameter in the test configuration page.

Tenant ID

Specify the Directory ID of the Azure AD tenant to which the target subscription belongs. To know how to determine the Directory ID, refer to Configuring the eG Agent to Monitor the Microsoft Azure App Service

Client ID and Client Password

The eG agent communicates with the target Microsoft Azure Subscription using Java API calls. To collect the required metrics, the eG agent requires an Access token in the form of an Application ID and the client secret value. To know how to determine the Application ID and the key, refer to Configuring the eG Agent to Monitor the Microsoft Azure App Service. Specify the Application ID of the created Application in the Client ID text box and the client secret value in the Client Password text box.

Proxy Host

In some environments, all communication with the Azure cloud be routed through a proxy server. In such environments, you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics. To enable metrics collection via a proxy, specify the IP address of the proxy server and the port at which the server listens against the Proxy Host and Proxy Port parameters. By default, these parameters are set to none, indicating that the eG agent is not configured to communicate via a proxy, by default.

Proxy Username, Proxy Password and Confirm Password

If the proxy server requires authentication, then, specify a valid proxy user name and password in the Proxy Username and Proxy Password parameters, respectively. Then, confirm the password by retyping it in the Confirm Password text box.

Detailed Diagnosis

To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

  • The eG manager license should allow the detailed diagnosis capability
  • Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.
Measures made by the test:
Measurement Description Measurement Unit Interpretation

Provisioning state

Indicates the current provisioning status of this vault.

 

The values reported by this measure and its numeric equivalents are mentioned in the table below:

Measure Value Numeric Value
Succeeded 1
Updating 2
Error 3
Unknown 0

Note:

By default, this measure reports the Measure Values listed in the table above to indicate the current provisioning status of a vault. In the graph of this measure however, the same is represented using the numeric equivalents only.

Use the detailed diagnosis of this measure to know the location, URI, family, and configuration of the vault.

Total API hits

Indicates the total number of times this vault has been accessed via the REST API.

Number

Azure Key Vault supports JSON formatted requests and responses. Requests to the Azure Key Vault are directed to a valid Azure Key Vault URL using HTTPS with some URL parameters and JSON encoded request and response bodies.

Average latency

Indicates the average time taken by this vault to respond to REST API requests.

Secs

A high value is indicative of slowness in processing the API requests.

Volume of operations

Indicates the total volume of operations in this vault.

Number

Compare the value of this measure across vaults to know which vault has the maximum operational workload.

Secret hit

Indicates the number of times secrets have been accessed from this vault.

Number

 

Vault hits

Indicates the number of times this vault has been accessed.

Number

If the value of this measure is abnormally high for any vault, then compare the value of the Secret hit, Key hit, Certificate hit, and Auxillary hit measures of that vault to know what type of requests are contributing to its workload - requests for secrets? keys? certificates? or others?

Key hits

Indicates the number of times keys in this vault have been accessed.

Number

 

Certificate hits

Indicates the number of times certificates in this vault have been accessed.

Number

 

Auxilliary hits

Indicates the number of times this vault has been accessed for contents other than keys, secrets, or certificates.

Number

 

Vault latency

Indicates the time that elapsed between when this vault was hit and when it responded.

Secs

A high value is indicative of a slow vault. If the value of this measure is abnormally high for any vault, then compare the value of the Key latency, Certificate latency, Secret latency, and Auxillary latency measures to know what type of requests are the most latent - requests for keys? certificates? secrets? or others?

Secret latency

Indicates the average time that elapsed between when a secret in this vault was hit and when that secret was returned.

Secs

Ideally, the value of this measure should be low. A high value implies that the vault is taking too long to process requests for secrets.

Key latency

Indicates the average time that elapsed between when a key in this vault was hit and when that key was returned.

Secs

Ideally, the value of this measure should be low. A high value implies that the vault is taking too long to process requests for keys.

Certificate latency

Indicates the average time that elapsed between when a certificate in this vault was hit and when that certificate was returned.

Secs

Ideally, the value of this measure should be low. A high value implies that the vault is taking too long to process requests for certificates.

Auxilliary latency

Indicates the average time that elapsed between when any other content in this vault was hit and when that content was returned.

Secs

Ideally, the value of this measure should be low. A high value implies that the vault is taking too long to process requests for contents other than keys/secrets/certificates.

Successful operations

Indicates the number of operations performed in this vault that succeeded.

Number

Ideally, the value of this measure should be equal or close to the value of the Volume of operations measure.

Failed operations

Indicates the number of operations performed in this vault that failed.

Number

Ideally, the value of this measure should be 0 or much lower than the value of the Volume of operations measure.

Other operations

Indicates the number of other operations that were performed in this vault.

Number