Monitoring Apache Kafka Message System
eG Enterprise offers a special-purpose monitoring model for the Apache Kafka Message System to monitor the status and overall performance of the target Apache Kafka Message System.
Figure 1 depicts the layer model of an Apache Kafka Message System.
Figure 1 : Layer model for Apache Kafka Message System
Every layer in the Figure 1 is mapped to various tests to determine the critical statistics related to the performance of the target Apache Kafka Message System. Using the metrics reported by the tests, administrators can find accurate answers for the following performance queries:
- Is the time taken for each fetch request between the consumer and the broker too long?
- Is there any lag noticed between the Kafka producers and consumers?
- What is the status of the Active controller - Is it idle?
- Is the coordinator group in stable condition?
- Is the average request latency too high for the broker's connection to the producer?
- Is there any connectivity issues between the producers, consumers and brokers?
- Were many records dropped due to buffer exhaustion during the producer instances?
- Is there any issues in the broker's connection to the Zookeeper?
Since the Operating System, Application Processes, Windows Service and TCP layers have been elaborately discussed in Monitoring Unix and Windows Servers document, the tests mapped to the Network layer have been elaborately discussed in Monitoring Cisco Router document, the tests mapped to the Application Transactions layer have been elaborately discussed in Monitoring JBoss EAP Serversdocument and the tests mapped to the JVM layer have already been discussed in Monitoring Java Applications document, the sections to come will discuss the other layers in detail.
