Elasticsearch ThreadPool Test

Thread pools are a collection of threads which are made available to perform specific tasks in the Elasticsearch cluster. A single Elasticsearch node holds multiple thread pools for different operations such as search, indexing, bulk operations, and more. The main goal of the thread pools is to make the memory management easier by managing the life-cycles of the threads while executing large number of requests. The thread pools are associated with queues to deal with the pending requests. When no free threads are available in the thread pool, the thread pool buffers the requests to the associated queue. Once the threads become free, the pending requests in the queue will be processed. This means that if enough free threads are not available in a pool, request processing will slowdown. Moreover, if the associated queues are also full, incoming requests will be rejected and errors will be thrown. As a result, search experience of users will be adversely impacted. To proactively detect and eliminate processing bottlenecks and to assure users of a satisfactory search experience, administrators should keep an eye on thread usage within pools. This is where the Elasticsearch ThreadPool test helps!

This test auto-discovers the thread pools in the Elasticsearch cluster, and sheds light on the current load and usage of threads in each thread pool. In the process, administrators can figure out whether the pools are sized commensurate to the load. Improper sizing of pools is thus brought to light. The test also reports the number of threads buffered to the queues. This enables administrators to quickly capture a sudden/consistent increase in queue size, which in turn may impact request processing.

Target of the test : An Elasticsearch Cluster

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for each thread pool in the target Elasticsearch cluster.

Configurable parameters for the test
Parameter Description

Test Period

How often should the test be executed.

Host

The IP address of the host for which this test is to be configured.

Port

The port number at which the target host being monitored listens. By default, this is set to 9200.

Cloud Instance URL

By default, the is set to none. If the target Elasticsearch cluster is hosted on the cloud environment, then you need to specify the URL of the cluster on cloud against this parameter. For example: 64bd966328067fd89e0c9b4c3bb8b042.us-east-1.aws.found.io. When the cloud URL is specified, the eG agent will use the cloud URL to monitor the target cluster rather than using the host specified in the Host text box.

Elastic Search User and Elastic Search Password

By default, the Elastic Search User and Elastic Search Password parameters are set to none indicating that the eG agent doesn't require authentication to collect metrics from the Elasticsearch cluster. If authentication is required to access the target Elasticsearch cluster, then specify the valid credentials against these parameters.

Confirm Password

Confirm the Elastic Search Password by retyping it in the Confirm Password text box.

SSL

By default, the SSL flag is set to No. If the Elasticsearch cluster is SSL-enabled by default or hosted on the cluster, then set this flag to Yes. This indicates that the eG agent will communicate with the target cluster via HTTPS by default.

Measurements made by the test
Measurement Description Measurement Unit Interpretation

Thread utilization

Indicates the percentage of threads in this thread pool that are currently in use.

Percent

This measure is computed using the following formula:

Thread utilization=(Active threads / Total threads)X100

A high value is indicative of a busy pool. A value close to 100% indicates excessive utilization of threads in a pool. If the value of this measure grows closer to 100% over time, it indicates that the pool is rapidly running out of threads to service the search requests. In such a case, administrators may have to fine-tune the thread pool size.

Active threads

Indicates the number of threads in this pool that are currently active in servicing the requests.

Number

This measure serves as a good indicator of the workload of the thread pool.

Comparing the value of this measure across the thread pools helps administrators identify the thread pool that is processing the maximum number of search requests.

Queued threads

Indicates the number of threads queued for processing in this pool.

Number

A low value is desired for this measure. A high value or a sudden increase in this value may indicate that the thread pool is unable to service requests as they come in. This can cause request processing delays.

Total threads

Indicates the maximum number of active threads that this pool can contain.

Number

 

Rejected threads

Indicates the number of threads that were rejected from this pool.

Number

Ideally, the value of this measure should be 0. If a non-zero value is reported, it implies that one/more requests were rejected owing to the lack of free threads in the pool. Administrators may then have to increase the size of the thread pool or queue in order to avoid rejections/errors.