Exadata Cell Disk IO Test

When database performance issues are related to I/O load on the Exadata storage servers, typically there will be increased latencies in the I/O-related wait events, and increased database time in the User I/O or System I/O wait classes.

A cell disk with a processing bottleneck will not be able to process user requests for data quickly, thereby causing prolonged delays in data access for users. Similarly, a cell disk that is overloaded will not be able to perform at peak capacity, thus affecting the user experience with the storage server. Administrators hence have to continuously track the load on and the processing speed of each of the cell disks, so that potential overload conditions and probable processing delays can be detected proactively and pre-emptively treated. The Exadata Cell Disk IO test helps administrators with this.

This test monitors the level of traffic on each cell disk created on an Oracle Exadata Storage Server, and helps isolate irregularities in load balancing across the cell disks. Alongside, the test also helps identify which cell disk is experiencing processing bottlenecks (if any), so that the bottleneck can be cleared before users complain of slowdowns.

Target of the test : Oracle Exadata Storage Server

Agent deploying the test : A remote agent

Outputs of the test : One set of results for each cell disk on the target Oracle Exadata Storage Server being monitored

Configurable parameters for the test
Parameter Description

Test period

How often should the test be executed

Host

The IP address of the host for which this test is to be configured.

Port

The port number at which the specified host listens. By default, this is NULL.

Username, Password and Confirm Password

By default, this test uses the Cell Control Command-Line Interface (CellCLI) to pull out the required metrics. To use the CLI, the test first needs to connect to the target storage server via SSH, and then run commands using CLI. For running the commands, this test requires the credentials of a cellmonitor user. Specify the login credentials of such a user in the User and Password text boxes and confirm the Password by retyping it in the Confirm Password text box.

SSH Port

This test uses the Cell CLI to pull metrics from the target Oracle Exadata Storage Server. To run the CLI commands, this test first needs to establish an SSH connection with the target storage server. To enable the test to establish this connection, specify the SSH Port here.

Timeout

 Specify the time duration for which this test should wait for a response from the storage system in the Timeout text box. By default, this is 120 seconds.

Measurements made by the test
Measurement Description Measurement Unit Interpretation

Large blocks read rate

Indicates the rate at which data was read in large blocks from this cell disk.

MB/sec

These measures are a good indicator of read I/O processing ability of the cell disks.

Compare the value of these measures across the cell disks to figure out the cell disk that reads maximum large blocks/small blocks.

Small blocks read rate

Indicates the rate at which data was read in small blocks from this cell disk.

MB/sec

Large blocks write rate

Indicates the rate at which data was written in large blocks to this cell disk.

MB/sec

These measures are a good indicator of write I/O processing ability of the cell disks.

Compare the value of these measures across the cell disks to figure out the cell disk that reads maximum large blocks/small blocks.

 

Small blocks write rate

Indicates the rate at which data was written in small blocks to this cell disk.

MB/sec

Scrubbing job read rate

Indicates the rate at which data was read from this cell disk by the scrubbing job.

MB/sec

Scrub IO - occurs when Oracle Exadata System Software automatically inspects and repairs the hard disks. Scrub I/O is performed periodically when the hard disks are idle, and mostly results in large disk reads, which should be throttled automatically if the disk becomes I/O bound.

IO error rate

Indicates the number of I/O errors recorded for this cell disk per minute.

Errors/min

Ideally, the value of this measure should be zero.

Compare the value of this measure across cell disks to identify the cell disk that is prone to errors.

Large block read requests

Indicates the number of read requests to read large blocks from this cell disk per second.

Requests/sec

Compare the value of these measures across the cell disks to figure out the cell disk that processes maximum read requests to read large blocks/small blocks.

Small block read requests

Indicates the number of read requests to read small blocks from this cell disk per second.

Requests/sec

Large block write requests

Indicates the number of write requests to write large blocks to this cell disk per second.

Requests/sec

Compare the value of these measures across the cell disks to figure out the cell disk that processes maximum write requests to write large blocks/small blocks.

 

Small block write requests

Indicates the number of write requests to write small blocks to this cell disk per second.

Requests/sec

Scrubbing job read requests

Indicates the number of requests to read data from from this cell disk by the scrubbing job per second.

Requests/sec

 

Average latency for Large block reads

Indicates the average time taken to read large blocks from this cell disk per request.

Milliseconds/request

A low value is desired for this measure. A sudden/gradual increase in the value of this measure indicates that the I/O processing ability of the cell disk is on decline. Administrators need to analyze the reason behind such issues and rectify them at the earliest.

Average latency for Small block reads

Indicates the average time taken to read small blocks from this cell disk per request.

Milliseconds/request

A low value is desired for this measure. A sudden/gradualincrease in the value of this measure indicates that the I/O processing ability of the cell disk is on decline. Administrators need to analyze the reason behind such issues and rectify them at the earliest.

Average latency for Large block writes

Indicates the average time taken to write large blocks to this cell disk per request.

Milliseconds/request

A low value is desired for this measure. A sudden/gradual increase in the value of this measure indicates that the I/O processing ability of the cell disk is on decline. Administrators need to analyze the reason behind such issues and rectify them at the earliest.

Average latency for Small block writes

Indicates the average time taken to write small blocks to this cell disk per request.

Milliseconds/request

A low value is desired for this measure. A sudden/gradual iincrease in the value of this measure indicates that the I/O processing ability of the cell disk is on decline. Administrators need to analyze the reason behind such issues and rectify them at the earliest.

Device utilization

Indicates the percentage of disk resources utilized for this cell disk.

Percent

A high value indicates that this cell disk is utilizing the maximum of disk resources.

Compare the value of this measure across cell disks to figure out the cell disk that is utilizing maximum disk resources.

Device utilization by large request

Indicates the percentage of disk resources utilized by large requests for this cell disk.

Percent

Compare the value of this measure across cell disks to figure out the cell disk that is utilizing the maximum of disk resources for processing large requests.

Device utilization by small request

Indicates the percentage of disk resources utilized by small requests for this cell disk.

Percent

Compare the value of this measure across cell disks to figure out the cell disk that is utilizing the maximum of disk resources for processing small requests.