Azure Key Vault Test

Azure Key Vault is a cloud service for securely storing and accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic keys. With Azure Key Vault, you can also easily provision, manage, and deploy digital certificates for your network and enable secure communications for your applications.

Anybody with an Azure subscription can create and use key vaults. Typically, Key Vaults benefit developers, security administrators, and even an organization's adminstrator who manages other Azure services. Authorized users can create a vault, create/import a key/secret in the vault, and can also authorize users/applications to access the vault to manage or use its contents.

If any application's attempt to access a vault ends in failure, or if any application experiences slowness during vault access, then that application will not be able to use the passwords/keys/certificates stored in the vault as and when it needs it. As a result, application performance will deteriorate. To avoid this, administrators should continuously monitor the status of and service levels delivered by every vault configured for an Azure subscription, and rapidly isolate the vault experiencing errors / failures / processing bottlenecks. Likewise, if a certificate stored by a vault expires, then any business-critical application using that certificate may be rendered inaccessible to users. To avoid this, administrators should track certificate validity continuously, identify active certificates, expired certificates, and the ones that are nearing expiry, and initiate measures to increase (if needed) the life of any business-critical certificate. With the Azure Key Vault test, administrators can achieve all of the above! 

This test auto-discovers the Key Vaults configured for an Azure subscription. For each vault so discovered, this test then tracks the provisioning status of that vault, and notifies administrators if errors are noticed during provisioning. The test also tracks the hits/requests to each vault, and measures the time taken by that vault to service the requests. This way, the test pinpoints the precise vault that is slow in processing requests to it. Additionally, the test also indicates the exact type of requests that a vault is having trouble processing - requests for keys? secrets? passwords? certificates? or others?. This eases troubleshooting. Similarly, the test sends out alerts if a certificate expires, pinpoints the expired certificate, and thus enables administrators to promptly renew that certificate. The detailed diagnosis of the test also reveals when each active certificate is set to expire, so that administrators can proactively detect an impending certificate expiry, and take pre-emptive steps.

Note:

One of the key capabilities of the Azure Key Vault test is to track the status of certificates stored in each Key Vault, and report details of expired and active certificates. To enable the test to pull these metrics, you need to do the following, before attempting to configure this test:

  • Create a separate Access Policy for every Key Vault you want to monitor;
  • Configure each such policy to grant Certificate Get and List permissions to the Azure AD application you created for monitoring purposes;

To know how to achieve the above, refer to the Granting Get & List Permissions to the Azure AD Application for Monitoring Key Vault Certificates topic.

 

Target of the Test: A Microsoft Azure Subscription

Agent deploying the test: A remote agent

Output of the test: One set of results for each Azure Key Vault configured for every resource group in the target Azure Subscription

Configurable parameters for the test
Parameters Description

Test Period

How often should the test be executed.

Host

The host for which the test is to be configured.

Subscription ID

Specify the GUID which uniquely identifies the Microsoft Azure Subscription to be monitored. To know the ID that maps to the target subscription, do the following:

  1. Login to the Microsoft Azure Portal.

  2. When the portal opens, click on the Subscriptions option (as indicated by Figure 1).

    Figure 1 : Clicking on the Subscriptions option

  3. Figure 2 that appears next will list all the subscriptions that have been configured for the target Azure AD tenant. Locate the subscription that is being monitored in the list, and check the value displayed for that subscription in the Subscription ID column.

    Figure 2 : Determining the Subscription ID

  4. Copy the Subscription ID in Figure 2 to the text box corresponding to the SUBSCRIPTION ID parameter in the test configuration page.

Tenant ID

Specify the Directory ID of the Azure AD tenant to which the target subscription belongs. To know how to determine the Directory ID, refer to Configuring the eG Agent to Monitor a Microsoft Azure Subscription Using Azure ARM REST API.

Client ID, Client Password, and Confirm Password

To connect to the target subscription, the eG agent requires an Access token in the form of an Application ID and the client secret value. For this purpose, you should register a new application with the Azure AD tenant. To know how to create such an application and determine its Application ID and client secret, refer to Configuring the eG Agent to Monitor a Microsoft Azure Subscription Using Azure ARM REST API. Specify the Application ID of the created Application in the Client ID text box and the client secret value in the Client Password text box. Confirm the Client Password by retyping it in the Confirm Password text box.

Proxy Host and Proxy Port

In some environments, all communication with the Azure cloud be routed through a proxy server. In such environments, you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics. To enable metrics collection via a proxy, specify the IP address of the proxy server and the port at which the server listens against the Proxy Host and Proxy Port parameters. By default, these parameters are set to none, indicating that the eG agent is not configured to communicate via a proxy, by default.

Proxy Username, Proxy Password and Confirm Password

If the proxy server requires authentication, then, specify a valid proxy user name and password in the Proxy Username and Proxy Password parameters, respectively. Then, confirm the password by retyping it in the Confirm Password text box.

Detailed Diagnosis

To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

  • The eG manager license should allow the detailed diagnosis capability
  • Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.
Measures made by the test:
Measurement Description Measurement Unit Interpretation

Provisioning state

Indicates the current provisioning status of this vault.

 

The values reported by this measure and its numeric equivalents are mentioned in the table below:

Measure Value Numeric Value
Succeeded 1
Updating 2
Error 3
Unknown 0

Note:

By default, this measure reports the Measure Values listed in the table above to indicate the current provisioning status of a vault. In the graph of this measure however, the same is represented using the numeric equivalents only.

Use the detailed diagnosis of this measure to know the location, URI, family, and configuration of the vault.

Total API hits

Indicates the total number of times this vault has been accessed via the REST API.

Number

Azure Key Vault supports JSON formatted requests and responses. Requests to the Azure Key Vault are directed to a valid Azure Key Vault URL using HTTPS with some URL parameters and JSON encoded request and response bodies.

Average latency

Indicates the average time taken by this vault to respond to REST API requests.

Secs

A high value is indicative of slowness in processing the API requests.

Volume of operations

Indicates the total volume of operations in this vault.

Number

Compare the value of this measure across vaults to know which vault has the maximum operational workload.

Secret hit

Indicates the number of times secrets have been accessed from this vault.

Number

 

Vault hits

Indicates the number of times this vault has been accessed.

Number

If the value of this measure is abnormally high for any vault, then compare the value of the Secret hit, Key hit, Certificate hit, and Auxillary hit measures of that vault to know what type of requests are contributing to its workload - requests for secrets? keys? certificates? or others?

Key hits

Indicates the number of times keys in this vault have been accessed.

Number

 

Certificate hits

Indicates the number of times certificates in this vault have been accessed.

Number

 

Auxilliary hits

Indicates the number of times this vault has been accessed for contents other than keys, secrets, or certificates.

Number

 

Vault latency

Indicates the time that elapsed between when this vault was hit and when it responded.

Secs

A high value is indicative of a slow vault. If the value of this measure is abnormally high for any vault, then compare the value of the Key latency, Certificate latency, Secret latency, and Auxillary latency measures to know what type of requests are the most latent - requests for keys? certificates? secrets? or others?

Secret latency

Indicates the average time that elapsed between when a secret in this vault was hit and when that secret was returned.

Secs

Ideally, the value of this measure should be low. A high value implies that the vault is taking too long to process requests for secrets.

Key latency

Indicates the average time that elapsed between when a key in this vault was hit and when that key was returned.

Secs

Ideally, the value of this measure should be low. A high value implies that the vault is taking too long to process requests for keys.

Certificate latency

Indicates the average time that elapsed between when a certificate in this vault was hit and when that certificate was returned.

Secs

Ideally, the value of this measure should be low. A high value implies that the vault is taking too long to process requests for certificates.

Auxilliary latency

Indicates the average time that elapsed between when any other content in this vault was hit and when that content was returned.

Secs

Ideally, the value of this measure should be low. A high value implies that the vault is taking too long to process requests for contents other than keys/secrets/certificates.

Successful operations

Indicates the number of operations performed in this vault that succeeded.

Number

Ideally, the value of this measure should be equal or close to the value of the Volume of operations measure.

Failed operations

Indicates the number of operations performed in this vault that failed.

Number

Ideally, the value of this measure should be 0 or much lower than the value of the Volume of operations measure.

Other operations

Indicates the number of other operations that were performed in this vault.

Number

 

Vault availability

Indicates whether/not this vault is available currently.

Percent

This measure reports the value 100 if the value is available, and the value 0 if it is not. Compare the value of this measure across vaults to identify the unavailable vaults.

Vault saturation

Indicates the percentage of this vault's capacity that has been utilized currently.

Percent

The storage capacity of an Azure Key Vault is limited by the total size of all the secrets, certificates, and keys stored within it. The maximum size of a single Key Vault is 25 KB (25,600 bytes) for standard vaults and 50 KB (51,200 bytes) for premium vaults.

A value over 75% for this measure is a cause for concern, as it indicates that the vault is about to exhaust its capacity. To ensure efficient usage and avoid reaching the maximum capacity, it is recommended to consider the size of the secrets and plan the organization and management of secrets within Azure Key Vault accordingly.

Active certificates

Indicates the number of active certificates in this vault.

Number

Use the detailed diagnosis of this measure to know which certificates are currently active and when are they set to expire. With the help of this information, administrators can quickly identify the certificates that are nearing expiry. If needed, they can change the certificate policy to increase its life and avoid expiry.

Expired certificates

Indicates the number of expired certificates in this vault.

Number

If this measure reports a non-zero value, then administrators can use the detailed diagnosis of this measure to identify the expired certificate. If required, the expired certificate can be renewed, so as to enable applications to continue using that certificate.