Apache Kafka Service Availability Test
The availability of Kafka denotes the ability of a Kafka cluster to continue producing and consuming messages without interruption, even when some brokers or components fail. If too many brokers are down or misconfigured, producers and consumers cannot connect or read/write messages, even if topics exist. Monitoring Kafka availability is necessary to ensure that messages are continuously produced, transmitted, and consumed without interruption, delay, or loss, even when failures occur in the system. This test helps administrators to monitor the availability of kafka brokers and also to monitor whether/not a successful transmission of messages is happening between the eG agent and kafka brokers.
Target of the test : Apache Kafka Message System
Agent deploying the test : An internal/remote agent
Outputs of the test : One set of results for the target Apache Kafka Message System.
| Parameter | Description |
|---|---|
|
Test period |
How often should the test be executed. |
|
Host |
The IP address of the target server that is being monitored. |
|
Port |
The port number through which the Apache Kafka Message System communicates. The default port is 9092. |
|
Bootstrap Servers |
Bootstrap Servers is a comma-separated list of host and port pairs that are the addresses of the Kafka brokers in a "bootstrap" Kafka cluster that a Kafka client connects to initially to bootstrap itself. By default, the BOOTSTRAP SERVERS text box will display the value as Host:Port. If a single Kafka Broker is associated with multiple Bootstraps in a Kafka cluster, then specify a comma-separated list of values as Host1:Port1, Host2:Port2,.... in this text box. If Kafka Message system is installed in a container, then the value will be displayed as Container ID:Port in this text box. |
|
Topic Name |
In Apache Kafka, a topic is a labeled channel that lets agents (producers and consumers) send and receive messages through the broker. Specify such Topic name in Topic Name text box. A topic is identified by a string name as the examples are eGtopicname1,eGtopicname2. |
|
Listener Name |
Specify the name of the listener service associated with the managed Apache Kafka Message System. By default, this parameter is set to PLAINTEXT. This means that by default the test will attempt to connect to and check the status of the 'default listener'. However, if the Apache Kafka Message System has been configured using a 'different listener', then you should select the value of the LISTENER NAME parameter. |
|
Truststore |
This parameter is applicable only if the target Apache Kafka Message System is SSL-enabled, if not, set this parameter to none. TrustStore is used to store certificates from Certified Authorities (CA) that verify and authenticate the certificate presented by the server in an SSL connection. Therefore, the eG agent should have access to the truststore where the certificates are stored to authenticate and connect with the target database and collect metrics. For this, first import the certificates into the following default location <eG_INSTALL_DIR>/lib/security/mytruststore.jks. As an example, to know how to import the certificate into the truststore, refer toPre-requisites for monitoring Oracle Cluster. Then, provide the truststore file name in this text box. For example: mytruststore.jks. By default, none is specified against this text box. |
|
Truststore Password and Confirm Password |
This parameter is applicable only if the target Apache Kafka Message System is SSL-enabled, if not, set this parameter to none. If a Truststore File name is provided, then, in this text box, provide the password that is used to obtain the associated certificate details from the Truststore File. By default, this parameter is set to none. Confirm the password by retyping the password in the Confirm Password text box. |
|
Keystore File |
This parameter is applicable only if the target Apache Kafka Message System is SSL-enabled, if not, set this parameter to none. Keystore contains the private keys for the certificates that the client can provide to the server upon request. eG agent requires access to the keystore where client certificate is stored to send that to the server so that the server validates the certificate against the one contained in its trustore. For this purpose, first create the client certificate in the following default location /opt/egurkha/jre/lib/security/egmqsslstore.jks. |
|
Keystore Password |
This parameter is applicable only if the target Apache Kafka Message System is SSL-enabled, if not, set this parameter to none. If a Keystore File name or file path is provided, then, in this text box, provide the password that is used to obtain the associated certificate details from the Keystore File. |
|
Confirm Password |
Confirm the Password for Keystore by retyping it here. |
|
SASL SSL Username |
This parameter is applicable only if the target Apache Kafka Message System is SSL-enabled, if not, set this parameter to none. SASL SSL Username parameter generally refers to the configuration of authentication and encryption protocols in Apache Kafka Message System, where SASL (Simple Authentication and Security Layer) is layered over SSL to provide secure, authenticated connections. |
|
SASL SSL Password and Confirm Password |
Specify the password in the SASL SSL Password text box for authenticated connection and confirm the password by retyping the password in the Confirm Password text box. |
|
SASL SSL Mechanism |
SASL SSL Mechanism in Apache Kafka Message System refers to the username and credential authentication methods supported through the SASL (Simple Authentication and Security Layer) framework, with communication secured via SSL/TLS encryption The two different types of authentication methods are SCRAM-SHA-256 and SCRAM-SHA-512. Select the required authentication method from the SASL SSL Mechanism list. |
|
DD Frequency |
Refers to the frequency with which detailed diagnosis measures are to be generated for this test. The default is 1:1. This indicates that, by default, detailed measures will be generated every time this test runs, and also every time the test detects a problem. You can modify this frequency, if you so desire. Also, if you intend to disable the detailed diagnosis capability for this test, you can do so by specifying none against DD frequency. |
|
Detailed Diagnosis |
To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option. The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:
|
| Measurement | Description | Measurement Unit | Interpretation | ||||||
|---|---|---|---|---|---|---|---|---|---|
|
Availability |
Indicates the availability of kafka brokers. |
|
A broker handles all requests from clients (produce, consume, and metadata) and keeps data replicated within the cluster. There can be one or more brokers in a cluster. The values reported by this measure and its numeric equivalents are mentioned in the table below:
Note: By default, this measure reports the Measure Values listed in the table above to indicate the availability of kafka brokers. The graph of this measure however, is represented using the numeric equivalents only. Use the detailed diagnosis of this measure to know the Kafka connectivity error details. |
||||||
|
Connection time |
Indicates the time taken for the eG agent to connect with kafka broker. |
Milliseconds |
A very low value of these measures is required for a successful transmission of messages to happen between the eG agent and kafka brokers.
|
||||||
|
Message processing time |
Indicates the processing time taken for transmission of messages between eG agent and kafka broker. |
Milliseconds |