Impala Daemon Current Completed Queries Test

This test tracks the currently executing queries on Impala Daemon server and determines the number of queries that have been waiting for a long time. If there are more number of queries in the waiting status, then the query needs to be explicitly closed for it to reach the "completed" status.

Target of the test : Apache Impala

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for each query typein the target Impala Daemon server.

Configurable parameters for the test
Parameter Description

Test period

How often should the test be executed.

Host

The IP address of the target server that is being monitored.

Port

The port number through which the Apache Impala communicates. The default port is 25000.

Impalad URL

Each daemon server has a different endpoint URL. You need to configure the eG agent with the endpoint URL of each daemon, so that the agent can access the URL and pull metrics on the health of the target daemon servers. Specify such URL in the Impalad URL text box . To know how to configure the eG agent with the API Endpoint URLs, refer to Configuring the eG agent with the API Endpoint URLs topic in Pre-requisites for Monitoring Apache Impala.

Impalad Authorization type

To access daemon server, you need to configure Digest authorization. Digest authorization is a method used to verify the identity of a user requesting access to API Endpoint URL of the daemon server. Digest authorization uses a combination of a username, password, and a unique message digest (a type of encrypted code). Specify such Digest authorization value in the Impalad Authorization type text box considering the below criteria:

  • If an Impalad URL is password protected (i.e., username and password configured through Cloudera Manager), then specify the value as digest_auth in the Impalad Authorization type text box. Here, digest_auth stands for Digest Authorization

  • If an Impalad URL is not password protected, then specify the value as no_auth in the Impalad Authorization type text box. Here, no_auth stands for No Authorization.

If multiple Impalad URLs are mentioned in the Impalad URL text box, then specify a comma-separated list of authorization type for each URL. For example, if there are two Impalad URLs, one URL without authorization and the other URL protected with authorization (username/password authorization), then specify Impalad Authorization type as: no_auth,digest_auth in the Impalad Authorization type text box. By default, Impalad Authorization type value is no_auth.

Impalad User, Impalad Password and Confirm Password

Once you enable web server access to the eG agent, make sure that 'password authentication' is also enabled. This is needed, so that the eG agent is able to access the built-in web server for each daemon in a safe, secure manner. Configure the credentials of such a user against Impalad User and Password text boxes. Confirm the password by retyping it in the Confirm Password text box. To know how to configure the eG agent with 'secure' access to the built-in web server, refer to Configuring the eG agent with 'secure' access to the built-in web server topic in Pre-requisites for Monitoring Apache Impala.

By default, Impalad user parameter is set to none.

  • If an Impalad URL is password protected (i.e., username and password configured through Cloudera Manager), then specify the configured username in the Impalad User text box.

  • If an Impalad URL is not password protected, then specify the value as none in the Impalad User text box.

  • An Impala daemon server can have multiple IP addresses. In this case, specify the corresponding URL parameter with a comma-separated list of endpoint URLs in the Impalad User textbox.

Statestored URL

Each daemon server has a different endpoint URL. You need to configure the eG agent with the endpoint URL of each daemon, so that the agent can access the URL and pull metrics on the health of the target daemon servers. Specify such URL in the Statestored URL text box . To know how to configure the eG agent with the API Endpoint URLs, refer to Configuring the eG agent with the API Endpoint URLs topic in Pre-requisites for Monitoring Apache Impala.

Statestored Authorization type

To access daemon server, you need to configure Digest authorization. Digest authorization is a method used to verify the identity of a user requesting access to API Endpoint URL of the daemon server. Digest authorization uses a combination of a username, password, and a unique message digest (a type of encrypted code). Specify such Digest authorization value in the Statestored Authorization type text box considering the below criteria:

  • If a Statestored URL is password protected (i.e., username and password configured through Cloudera Manager), then specify the value as digest_auth in the Statestored Authorization type text box. Here, digest_auth stands for Digest Authorization

  • If an Statestored URL is not password protected, then specify the value as no_auth in the Statestored Authorization type text box. Here, no_auth stands for No Authorization.

By default, Statestored Authorization type value is no_auth.

Statestored User, Statestored Password and Confirm Password

Once you enable web server access to the eG agent, make sure that 'password authentication' is also enabled. This is needed, so that the eG agent is able to access the built-in web server for each daemon in a safe, secure manner. Configure the credentials of such a user against Statestored User and Password text boxes. Confirm the password by retyping it in the Confirm Password text box. To know how to configure the eG agent with 'secure' access to the built-in web server, refer to Configuring the eG agent with 'secure' access to the built-in web server topic in Pre-requisites for Monitoring Apache Impala.

By default, Impalad user parameter is set to none.

  • If a Statestored URL is password protected (i.e., username and password configured through Cloudera Manager), then specify the configured username in the Statestored User text box.

  • If a Statestored URL is not password protected, then specify the value as none in the Statestored User text box.

Catalogd URL

Each daemon server has a different endpoint URL. You need to configure the eG agent with the endpoint URL of each daemon, so that the agent can access the URL and pull metrics on the health of the target daemon servers. Specify such URL in the Catalogd URL text box . To know how to configure the eG agent with the API Endpoint URLs, refer to Configuring the eG agent with the API Endpoint URLs topic in Pre-requisites for Monitoring Apache Impala.

Catalogd Authorization type

To access daemon server, you need to configure Digest authorization. Digest authorization is a method used to verify the identity of a user requesting access to API Endpoint URL of the daemon server. Digest authorization uses a combination of a username, password, and a unique message digest (a type of encrypted code). Specify such Digest authorization value in the Catalogd Authorization type text box considering the below criteria:

  • If a Catalogd URL is password protected (i.e., username and password configured through Cloudera Manager), then specify the value as digest_auth in the Catalogd Authorization type text box. Here, digest_auth stands for Digest Authorization

  • If an Catalogd URL is not password protected, then specify the value as no_auth in the Catalogd Authorization type text box. Here, no_auth stands for No Authorization.

By default, Statestored Authorization type value is no_auth.

Catalogd User, Catalogd Password and Confirm Password

Once you enable web server access to the eG agent, make sure that 'password authentication' is also enabled. This is needed, so that the eG agent is able to access the built-in web server for each daemon in a safe, secure manner. Configure the credentials of such a user against Catalogd User and Password text boxes. Confirm the password by retyping it in the Confirm Password text box. To know how to configure the eG agent with 'secure' access to the built-in web server, refer to Configuring the eG agent with 'secure' access to the built-in web server topic in Pre-requisites for Monitoring Apache Impala.

By default, Impalad user parameter is set to none.

  • If a Catalogd URL is password protected (i.e., username and password configured through Cloudera Manager), then specify the configured username in the Catalogd User text box.

  • If a Catalogd URL is not password protected, then specify the value as none in the Catalogd User text box.

DD Frequency

Refers to the frequency with which detailed diagnosis measures are to be generated for this test. The default is 1:1. This indicates that, by default, detailed measures will be generated every time this test runs, and also every time the test detects a problem. You can modify this frequency, if you so desire. Also, if you intend to disable the detailed diagnosis capability for this test, you can do so by specifying none against DD frequency.

Detailed Diagnosis

To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

  • The eG manager license should allow the detailed diagnosis capability
  • Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.
Measurements made by the test
Measurement Description Measurement Unit Interpretation

Executing queries

Indicates the total number of executing queries in the Impala Daemon server.

Number

 

Waiting queries

Indicates the number of waiting queries.

Number

Queries will be in the "waiting to be closed" stage if they are in the EXCEPTION state or if all the rows from the query have been read.

Ideally, the value of this measure should be very low. if value is high , then the query needs to be explicitly closed for it to be "completed".

In-flight queries

Indicates the number of In-flight queries.

Number

When queries 'time out', they are cancelled but not closed, i.e. the query will remain "in flight" with a CANCELLED status.

Exception queries

Indicates the number of exception queries.

Number

Queries will be in the "waiting to be closed" stage if they are in the EXCEPTION state.

Ideally, the value of this measure should be very low. if value is high , then cause needs to be checked using detailed diagnosis of this measure for resolution.

The detailed diagnosis of this measure provides details about the exception queries for Impala daemon like the Description, User, Default DB, Statement, State, Status, Start time, End time, Duration, Progress, Rows fetched, Query ID, Waiting, Executing and Waiting time.

Duration

Indicates the total number of queries executed duration.

Milliseconds

 

Fetched rows

Indicates the total number of queries fetched rows.

Number