Cluster Resource Pools Test

This test auto-discovers the resource pools/vApps under ESX clusters, and reports critical statistics pertaining to each pool.

Target of the test : A VMware vCenter server

Agent deploying the test : An internal agent

Outputs of the test : One set of results for every datacenter::cluster::resourcepool and/or datacenter::cluster::vApp combination managed by vCenter; ; to distinguish between vApps and resource pools, the eG monitoring console tags every discovered vApp as [vApp].

Configurable parameters for the test
Parameter Description

Test Period

How often should the test be executed.

Host

The host for which this test is to be configured.

Port

Refers to the port at which the specified host listens to.

VC User and VC Password

To connect to vCenter and extract metrics from it, this test should be configured with the name and password of a user with Administrator or Virtual Machine Administrator privileges to vCenter. However, if, owing to security constraints, you are not able to use the credentials of such users for test configuration, then you can configure this test with the credentials of a user with Read-only rights to vCenter. For this purpose, you can assign the ‘Read-only’ role to a local/domain user to vCenter, and then specify name and password of this user against the VC User and VC Password text boxes. The steps for assigning this role to a user on vCenter have been detailed in the Monitoring VMware Infrastructures chapter

vCenter servers terminate user sessions based on timeout periods. The default timeout period is 30 mins. When you stop an agent, sessions currently in use by the agent will remain open for this timeout period until vCenter times out the session. If the agent is restarted within the timeout period, it will open a new set of sessions. If you want the eG agent to close already existing sessions on vCenter before it opens new sessions, then, instead of the ‘Read-only’ user, you can optionally configure the VC User and VC Password parameters with the credentials of a user with permissions to View and Stop Sessions on vCenter. For this purpose, you can create a special role on vCenter, grant the View and Stop Sessions privilege (prior to vCenter 4.1, this was called the View and Terminate Sessions privilege) to this role, and then assign the new role to a local/domain user to vCenter. The steps for this have been discussed in the Monitoring VMware Infrastructures chapter.

Confirm Password

Confirm the password by retyping it in this text box.

SSL

By default, the vCenter server is SSL-enabled. Accordingly, the SSL flag is set to Yes by default. This indicates that the eG agent will communicate with the vCenter server via HTTPS by default.

Webport

By default, in most virtualized environments, vCenter listens on port 80 (if not SSL-enabeld) or on port 443 (if SSL-enabled) only. This implies that while monitoring vCenter, the eG agent, by default, connects to port 80 or 443, depending upon the SSL-enabled status of vCenter – i.e., if vCenter is not SSL-enabled (i.e., if the SSL flag above is set to No), then the eG agent connects to vCenter using port 80 by default, and if vCenter is SSL-enabled (i.e., if the ssl flag is set to Yes), then the agent-vCenter communication occurs via port 443 by default.  Accordingly, the Webport parameter is set to default by default. 

In some environments however, the default ports 80 or 443 might not apply. In such a case, against the Webport parameter, you can specify the exact port at which vCenter in your environment listens, so that the eG agent communicates with that port for collecting metrics from vCenter.

DD Frequency

Refers to the frequency with which detailed diagnosis measures are to be generated for this test. The default is 4:1. This indicates that, by default, detailed measures will be generated every time this test runs, and also every time the test detects a problem. You can modify this frequency, if you so desire. Also, if you intend to disable the detailed diagnosis capability for this test, you can do so by specifying none against DD frequency.

Detailed Diagnosis

To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

  • The eG manager license should allow the detailed diagnosis capability
  • Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.
Measurements made by the test
Measurement Description Measurement Unit Interpretation

Pool status

Indicates the current status of this resource pool.

 

If the status cannot be determined, then Unknown will be displayed here. If the pool is functioning normally, then this measure will return the value Normal. If the root resource pool does not have the capacity to meet the reservation of its children, then the status will be displayed as Overcommitted. One or more pools in the tree may have children whose reservations are greater than the pool is configured to support, then the status will be Inconsistent.

The numeric values that correspond to these measures are as follows:

State Value
Unknown 0
Normal 1
Overcommitted 2
Inconsistent 3

Note:

By default, this measure reports the Measure Values listed in the table above to indicate the status of each resource pool. The graph of this measure however, represents the status of a resource pool using the numeric equivalents - 1 to 3.

Child resource pools

Indicates the total number of child resource pools under this resource pool.

Number

 

VMs in pool

Indicates the total number of VMs in this resource pool.

Number

This number includes both the directly and indirectly assigned VMs. 

VMs powered on

Indicates the total number of powered-on VMs in this resource pool.

Number

This number includes both the directly and indirectly assigned VMs. 

Direct VMs in pool

Indicates the number of virtual machines that were directly assigned to this resource pool.

Number

 

Direct VMs powered on

Indicates the number of directly assigned virtual machines under this resource pool that is currently powered on.

Number

If the value of this measure is equal to the value of the Direct VMs in pool measure, it indicates that all VMs in the pool are in a powered-on state. If the values do not match, it indicates that one/more VMs are currently in a powered-off state.

CPU reserved for VMs in pool

The total amount of  CPU reserved by VMs in this pool.

Mhz

Use the detailed diagnosis of this measure to know how much CPU has been reserved by each VM.

Host CPU used by VMs

Indicates the percentage of physical CPU resources used by the VMs in this pool.

Percent

A high value of this measure could indicate excessive CPU usage by the child resource pools/virtual machines configured under this resource pool. By comparing the value of this measure across resource pools, you can quickly identify that resource pool which is draining the CPU resources of the ESX host.

The detailed diagnosis of this measure, if enabled, reveals the names and CPU usage of each VM in this pool, thus revealing which VM is CPU-intensive.

CPU capacity used

Indicates the CPU usage of the resource pool in Mhz

Mhz

The percent CPU usage measure serves as an effective indicator of how resource-intensive a particular resource pool is on a specific ESX server. However, for performing capacity planning or what-if analysis, the CPU usage of a pool measured in absolute terms would be more useful.  For instance, the physical CPU usage of a pool could be 30% on a particular ESX server – this means that the resource pool is consuming 30% of the total physical CPU capacity of that ESX server. If you are planning to migrate the resource pool to another ESX server, then it would be unwise to assume that the pool will only consume 30% of CPU on the other ESX server as well, as the percentage will vary depending upon the physical CPU resources that are available to the other ESX server. The absolute measure however will remain unchanged across ESX server. Therefore, to decide which ESX server a VM/resource pool is to be moved to, and to analyze the impact of this movement on the CPU resources of the new ESX host, you would require an absolute measure of CPU usage.    

CPU capacity in pool

Indicates the amount of CPU resources that is guaranteed available to the virtual machine or resource pool.

Mhz

Typically, the Reservation setting for a resource pool specifies the minimum acceptable amount of CPU or memory - not the amount you would like to have available.

Follow the guidelines mentioned below to define the Reservation setting:

  • Do not set the Reservation too high. A very high reservation can limit the number of virtual machines you can power on in a resource pool.
  • Always leave some headroom while reserving resource. Do not commit all resources. As you move closer to fully reserving all capacity in the system, it becomes increasingly difficult to make changes to reservations and to the resource pool hierarchy without violating admission control.
  • If you expect frequent changes to the total available resources, use Shares, not Reservation, to allocate resources fairly across resource pools.

Reserved resources are not wasted if they are not used. If the utilization is less than the reservation, the resources can be utilized by other running virtual machines.

CPU capacity reserved

Indicates the total amount of CPU resources that have been used to satisfy the reservation requirements of all descendants of this resource pool (includes both resource pools and virtual machines).

Mhz

CPU resources are typically reserved for powered-on virtual machines in a resource pool only. Virtual machines that are not currently running do not use any CPU resources.

Reserved resources are not wasted if they are not used. If the utilization is less than the reservation, the resources can be utilized by other running virtual machines.

CPU available in resource pool

Indicates the total amount of resources available to satisfy a reservation for a child resource pool.

Mhz

In the under-committed state, this value is limited by the capacity at the root node. In the overcommitted case, this could be higher since we do not perform the dynamic capacity checks.

Consider a resource pool with reservation=2GHz that is totally idle. It has 2GHz reserved, but it is not actually using any of its reservation. In such a case:

  • Other resource pools cannot reserve these 2GHz.
  • Other resource can use these 2GHz, that is, idle CPU reservations are not wasted.

CPU allocation used

Indicates the percentage of allocated CPU resources that are being used by this resource pool.

Percent

A high value of this measure indicates that one/more VMs in the pool are consuming the allocated CPU resources excessively.

CPU limit of pool

Indicates the limit beyond which this resource pool should not use CPU resources, even if there are resources available.

Mhz

If the Unlimited flag is enabled for a resource pool, then this measure will report the value Unlimited. This setting helps avoid wastage of idle resources.

This is typically used to ensure a consistent performance of virtual machines / resource pools independent of available resources. If set to -1, then there is no fixed limit on resource usage (only bounded by available resources and shares).

Memory reserved for VMs in pool

The total amount of  memory reserved by VMs in this pool.

MB

Use the detailed diagnosis of this measure to know how much memory has been reserved by each VM in the pool.

Memory consumed by VMs in pool

Indicates the total amount of memory consumed by the VMs in this pool.

MB

Use the detailed diagnosis of this measure to know how much memory has been consumed by each VM in the pool. This way, you can quickly identify memory-intensive VMs.

Memory used by running VMs

Indicates the amount of memory used by running VMs in this resource pool.

MB

 

Memory capacity

Indicates the amount of memory resources that is guaranteed available to the virtual machine or resource pool.

MB

Typically, the Reservation setting for a resource pool specifies the minimum acceptable amount of CPU or memory - not the amount you would like to have available.

Follow the guidelines mentioned below to define the Reservation setting:

  • Do not set the Reservation too high. A very high reservation can limit the number of virtual machines you can power on in a resource pool.
  • Always leave some headroom while reserving resource. Do not commit all resources. As you move closer to fully reserving all capacity in the system, it becomes increasingly difficult to make changes to reservations and to the resource pool hierarchy without violating admission control.
  • If you expect frequent changes to the total available resources, use Shares, not Reservation, to allocate resources fairly across resource pools.

Reserved resources are not wasted if they are not used. If the utilization is less than the reservation, the resources can be utilized by other running virtual machines.

Memory capacity reserved

Indicates the total amount of memory resources that have been used to satisfy the reservation requirements of all descendants of this resource pool (includes both resource pools and virtual machines).

MB

Memory resources are typically reserved for powered-on virtual machines in a resource pool only. Virtual machines that are not currently running do not use any memory resources.

Reserved resources are not wasted if they are not used. If the utilization is less than the reservation, the resources can be utilized by other running virtual machines.

Memory available in resource pool

Indicates the total amount of resources available to satisfy a reservation for a child resource pool.

MB

In the undercommitted state, this value is limited by the capacity at the root node. In the overcommitted case, this could be higher since we do not perform the dynamic capacity checks.

If the resource pool does not use the reserved memory, then remember the following:

  • Other resource pools cannot reserve these memory resources
  • Other resource pools can use these memory resources, that is, idle memory reservations are not wasted.

Free memory in pool

Indicates the percentage of free memory in this resource pool.

Percent

A consistent decrease in the value of this measure could indicate a potential memory contention in the pool. Compare the value of this measure across resource pools to isolate the resource pool with the least amount of free memory. 

Memory limit of pool

Indicates the limit beyond which this resource pool should not use memory resources, even if there are resources available.

MB

If the Unlimited flag is enabled for a resource pool, then this measure will report the value Unlimited. This setting helps avoid wastage of idle resources.

This is typically used to ensure a consistent performance of virtual machines / resource pools independent of available resources. If set to -1, then there is no fixed limit on resource usage (only bounded by available resources and shares).

Active memory

Indicates the amount of memory resources actively used from this pool.

MB

High memory consumption by a resource pool, besides eroding the physical memory resources of the host, can also impact the memory allocations to other resource pools/virtual machines; the lack of adequate memory to a VM can significantly strain its performance. 

Active memory used

Indicates the percentage of total configured or available memory in this pool.

Percent

Zero memory

Indicates the amount of memory that is zeroed out.

MB

 

Swap memory

Indicates the amount of memory that is swapped.

MB

ESX Server hosts use swapping to forcibly reclaim memory from a virtual machine when no vmmemctl driver is available because the vmmemctl driver:

  • Was never installed
  • Has been explicitly disabled
  • Is not running (for example, while the guest operating system is booting)
  • Is temporarily unable to reclaim memory quickly enough to satisfy current system demands

Standard demand-paging techniques swap pages back in when the virtual machine needs them.

Swap space must be reserved on disk for any unreserved virtual machine memory. This swap reservation is required to ensure the system is able to preserve virtual machine memory under any circumstances. In practice, only a small fraction of the swap space may actually be used.

Typically, swap space usage for each VM should be low. Since access from RAM is much faster than access from physical disk, excessive usage of swap memory will slow down the performance of a VM. Watch for VMs that are seeing higher swap usage and more swap reads and writes.

Memory overhead

Indicates the memory overhead incurred by this resource pool.

MB

ESX Server virtual machines can incur two kinds of memory overhead:

  • The additional time to access memory within a virtual machine.
  • The extra space needed by the ESX Server host for its own code and data structures, beyond the memory allocated to each virtual machine.

The ESX server’s memory virtualization ensures that the time overhead to memory accesses is minimal.

The memory space overhead, on the other hand, is composed of two other components:

  • A fixed system-wide overhead for the service console (in the case of ESX 3/3.5) and the VMkernel.
  • Additional overhead for each virtual machine

In addition, the space reserved for the virtual machine frame buffer and various virtualization data structures, also add to the memory overhead.

Addition of more virtual CPUs, allocation of more memory to the guests, and choosing to configure 32-bit guest operating systems instead of 64-bit ones, can go a long way in reducing this overhead. The ESX server also provides optimizations such as memory sharing to save up memory.

Balloon memory

Indicates the amount of memory used by memory control.

MB

The vmmectl driver that is installed on a virtual machine, emulates an increase or decrease in memory pressure on the guest operating system; this way, it forces the guest OS to place memory pages into its local swap file. This driver differs from the VMware swap file method as it forces the operating system to determine what memory it wishes to page. Once the memory is paged locally on the guest operating system, the free physical pages of memory may be reallocated to other guests. As the ESX hosts sees that memory demand has been reduced, it will instruct vmmemctl to “deflate” the balloon and reduce pressure on the guest OS to page memory.

The maximum amount of memory that can be reclaimed from a guest may be configured by modifying the “sched.mem.maxmemctl” advanced option.

If the memory reclaimed from a guest (i.e., the value of this measure) is very low, it indicates excessive memory usage by the guest. Under such circumstances, you might want to consider allocating more memory to the guest.

Shared memory

Indicates the amount of memory in the resource pool that is shared.

MB

 

Memory granted

Indicates the amount of memory that is granted to this resource pool.

MB