Storage Adapters - ESX Test

Using this test, you can monitor the current state and load on each storage adapter supported by the vSphere host, and can thus quickly and accurately isolate failed/overloaded adapters.

Target of the test : An ESX server host

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for every storage adapter

Configurable parameters for the test:
Parameter Description

Test Period

How often should the test be executed

Host

The host for which the test is to be configured

Port

The port at which the specified HOST listens. By default, this is NULL.

ESX User and ESX Password

In order to enable the test to extract the desired metrics from a target ESX server, you need to configure the test with an ESX USER and ESX PASSWORD. The user credentials to be passed here depend upon the mechanism used by the eG agent for auto-discovering the VMs on the target vSphere server and monitoring the server and its VMs. These discovery/monitoring methodologies and their corresponding configuration requirements have been discussed hereunder:

  • Discovering and monitoring by directly connecting to the target vSphere server: Starting with ESX server 3.0, a VMware ESX server offers a web service interface using which the eG agent discovers the guest operating systems on a physical ESX host. The VMware VI SDK is used by the agent to implement the web services interface. To use this interface for discovering the VMs and for monitoring, the eG agent should directly connect to the monitored vSphere/ESX server as an ESX USER with root privileges. However, if, owing to security constraints, you cannot use root user permissions, you can alternatively configure the tests with the credentials of a user who has been assigned the following permissions:

    • Diagnostics

    • TerminateSession

    To see how you can create such a user on the ESX server, refer to theCreating a Special Role on an ESX Server and Assigning the Role to a New User topic

  • Discovering and monitoring using vCenter: By default, the eG agent connects to each ESX server and discovers the VMs executing on it. While this approach scales well, it requires additional configuration for each server being monitored. For example, separate user accounts may need to be created on each server for accessing VM details. While monitoring large virtualized installations however, the agents can be optionally configured to perform guest discovery using the VM information already available in vCenter. The same vCenter can also be used to monitor the vSphere server and its VMs. In this case therefore, the ESX USER and ESX PASSWORD that you specify should be that of an Administrator or Virtual Machine Administrator in vCenter. However, if, owing to security constraints, you prefer not to use the credentials of such users, then, you can create a special role on vCenter with the following privileges:

    • Diagnostics

    • Change settings

    • View and stop sessions

    To know how to grant the above-mentioned permissions to a vCenter user, refer to Creating a Special Role on vCenter and Assigning the Role to a New User .

    If the ESX server for which this test is being configured had been discovered via vCenter, then the eG manager automatically populates the ESX USERand ESX PASSWORD text boxes with the vCenter user credentials using which the ESX discovery was performed.

Confirm Password

Confirm the specified ESX PASSWORD by retyping it here.

SSL

By default, the ESX server is SSL-enabled. Accordingly, the SSL flag is set to Yes by default. This indicates that the eG agent will communicate with the ESX server via HTTPS by default. On the other hand, if the eG agent has been configured to use the VMPerl API or CLI for monitoring (i.e., if the ESX USER parameter is set to none), then the status of the SSL flag is irrelevant.

Like the ESX sever, the vCenter is also SSL-enabled by default. If you have chosen to use the vCenter for monitoring all the ESX servers in your environment, then you have to set the SSL flag to Yes.

Webport

By default, in most virtualized environments, the ESX server and vCenter listen on port 80 (if not SSL-enabled) or on port 443 (if SSL-enabled). This implies that while monitoring an SSL-enabled ESX server directly, the eG agent, by default, connects to port 443 of the ESX server to pull out metrics, and while monitoring a non-SSL-enabled ESX server, the eG agent connects to port 80. Similarly, while monitoring an ESX server via an SSL-enabled vCenter, the eG agent connects to port 443 of vCenter to pull out the metrics, and while monitoring via a non-SSL-enabled vCenter, the eG agent connects to port 80 of vCenter. Accordingly, the WEBPORT parameter is set to 80 or 443 depending upon the status of the SSL flag.  In some environments however, the default ports 80 or 443 might not apply. In such a case, against the WEBPORT parameter, you can specify the exact port at which the ESX server or vCenter in your environment listens so that the eG agent communicates with that port.

Virtual Center

If the eG manager had discovered the target ESX server by connecting to vCenter, then the IP address of the vCenter server used for discovering this ESX server would be automatically displayed against the VIRTUAL CENTER parameter; similarly, the ESX USER and ESX PASSWORD text boxes will be automatically populated with the vCenter user credentials, using which ESX discovery was performed.

If this ESX server has not been discovered using vCenter, but you still want to discover the guests on the ESX server via vCenter, then select the IP address of the vCenter host that you wish to use for guest discovery from the VIRTUAL CENTER list. By default, this list is populated with the IP address of all vCenter hosts that were added to the eG Enterprise system at the time of discovery. Upon selection, the ESX USERand ESX PASSWORD that were pre-configured for that vCenter server will be automatically displayed against the respective text boxes.

On the other hand, if the IP address of the vCenter server of interest to you is not available in the list, then, you can add the details of the vCenter server on-the-fly, by selecting the Other option from the VIRTUAL CENTER list. This will invoke the ADD VCENTER SERVER DETAILS page. Refer to Adding the Details of a vCenter Server for VM Discoverysection to know how to add a vCenter server using this page. Once the vCenter server is added, its IP address, ESX USER, and ESX PASSWORD will be displayed against the corresponding text boxes.

On the other hand, if you want the eG agent to behave in the default manner -i.e., communicate with each ESX server for monitoring and VM information - then set the VIRTUAL CENTER parameter to ‘none’.

Report LUNs Details in DD

The detailed diagnosis of the Number of LUNs mapped to the storage adapter measure lists the LUNs mapped to a storage adapter. In large virtualized environments, a storage adapter may be mapped to numerous LUNs. As a result, the detailed diagnosis of the Number of LUNs mapped to the storage adapter measure may appear cluttered. Also, these detailed metrics will consume a considerable amount of storage space. This is why, by default, this test does not report detailed diagnostics for the Number of LUNs mapped to the storage adapter measure. Accordingly, this flag is set to No by default. If you have a well-sized, well-tuned eG database, then you can set this flag to Yes, and enable the collection of detailed metrics for the Number of LUNs mapped to the storage adapter measure.

Report Target Details in DD

The detailed diagnosis of the Number of targets for the storage adapter measure lists the names of the targets for a storage adapter. In large virtualized environments, a storage adapter may have numerous targets. As a result, the detailed diagnosis of the Number of targets for the storage adapter measure may appear cluttered. Also, these detailed metrics will consume a considerable amount of storage space. This is why, by default, this test does not report detailed diagnostics for the Number of targets for the storage adapter measure. Accordingly, this flag is set to No by default. If you have a well-sized, well-tuned eG database, then you can set this flag to Yes, and enable the collection of detailed metrics for the Number of targets for the storage adapter measure.

Report Path Details in DD

The detailed diagnosis of the Number of paths to the storage adapter measure lists the paths using which the target vSphere host communicates with a storage adapter. In large virtualized environments, a vSphere host can access a storage adapter using multiple paths. As a result, the detailed diagnosis of the Number of paths to the storage adapter measure may appear cluttered. Also, these detailed metrics will consume a considerable amount of storage space. This is why, by default, this test does not report detailed diagnostics for the Number of paths to the storage adapter measure. Accordingly, this flag is set to No by default. If you have a well-sized, well-tuned eG database, then you can set this flag to Yes, and enable the collection of detailed metrics for the Number of paths to the storage adapter measure.

DD Frequency

Refers to the frequency with which detailed diagnosis measures are to be generated for this test. The default is 1:1. This indicates that, by default, detailed measures will be generated every time this test runs, and also every time the test detects a problem. You can modify this frequency, if you so desire. Also, if you intend to disable the detailed diagnosis capability for this test, you can do so by specifying none against DD FREQUENCY.

Detailed Diagnosis

To make diagnosis more efficient and accurate, eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

  • The eG manager license should allow the detailed diagnosis capability

  • Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.

Measurements made by the test
Measurement Description Measurement Unit Interpretation

Average write requests rate:

Indicates the average number of write commands issued per second on this storage adapter during the collection interval.

Commands/Sec

This is a good indicator of the workload on each storage adapter. Comparing the value of these measures across adapters will reveal the overloaded storage adapter.

Excessive loading on a storage path can cause adapter overloads. Using the detailed diagnosis of these measures, you can compare the rate at which read requests and write requests were issued to all the storage paths via which the vSphere host is communicating with the adapter. Overloaded storage paths can thus be isolated.

Average read requests rate:

Indicates the average number of read commands issued per second on this storage adapter during the collection interval.

Commands/Sec

Write latency:

Indicates the average amount of time taken for a write on this storage adapter.

Secs

High latencies are often indicative of a processing bottleneck on the storage adapter. Excessive loading of the adapter, or of the path using which the host communicates with the adapter could also cause read/write slowdowns.

Compare the value of these measures across adapters to determine which adapter is experiencing the maximum latency.

Read latency:

Indicates the average amount of time taken for a read on this storage adapter.

Secs

Read rate:

Indicates the rate at which commands are read from this storage adapter.

MB/Sec

This is a good indicator of the workload on each storage adapter. Comparing the value of these measures across adapter will reveal the overloaded storage adapter, thus shedding light on irregularities (if any) in load balancing. 

Write rate:

Indicates the rate at which commands are written to this storage adapter.

MB/Sec

Average commands issued:

Indicates the average number of commands issued per second on this storage adapter during the collection interval.

Commands/Sec

Status:

Indicates the current state of this storage adapter.

 

This measure can report any of the following values to indicate the state of a storage adapter:

  • Online
  • Offline
  • Fault
  • Unknown

The numeric values that correspond to the above-mentioned states are as follows:

State Numeric Value

Online

0

Offline

1

Fault

2

Unknown

3

Note:

By default, this measure reports the above-mentioned States while indicating the state of the storage adapter. However, the graph of this measure will represent adapter states using the corresponding numeric equivalents mentioned in the table above.

Number of LUNs mapped to the storage adapter:

Indicates the number of LUNs that are communicating with this storage adapter.

Number

Use the detailed diagnosis of this measure, if enabled, to know which LUNs are mapped to the storage adapter.

Number of targets for the storage adapter:

Indicates the number of targets that are communicating with this storage adapter.

Number

Use the detailed diagnosis of this measure, if enabled, to know which are the targets for this storage adapter.

Number of paths to the storage adapter:

Indicates the number of paths using which the vSphere host is communicating with this storage adapter.

Number

Use the detailed diagnosis of this measure, if enabled, to know using which paths this vSphere host communicates with the storage adapter.

 

r.

Total latency:

Indicates the total amount of time taken to read from and write into this storage adapter from the perspective of guest operating system.

Secs

A high value or a consistent rise in the value of this measure is a cause for concern, as it indicates a current/potential I/O processing bottleneck with the storage adapter.

Throughput:

Indicates the rate at which the data is read from or written into the storage adapter.

 

MB/Sec

Ideally, the value of this measure should be high. A sudden or steady drop in this value could indicate a gradual deterioration in the performance of the storage adapter, probably owing to a processing bottleneck.