Kafka Broker Topic Test

A Kafka cluster consists of one or more servers (Kafka brokers). Each Broker can have one or more topics. A topic is a category or feed name to which records are published. Kafka topics are divided into a number of partitions, each partition can be placed on a single or separate machine to allow for multiple consumers to read from a topic in parallel.

This test helps the administrators to monitor the request from producers, consumers and broker followers to the brokers, ensuring that the target Apache Kafka Message System is communicating efficiently and resolve potential contentions, if any.

Target of the test : Apache Kafka Message System

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for the target Apache Kafka Message System.

Configurable parameters for the test
Parameter	Description
Test period	How often should the test be executed.
Host	The IP address of the target server that is being monitored.
Port	The port number through which the Apache Kafka Message System communicates. The default port is 9092.
JMX Remote Port	This parameter appears only if the Measurement Mode is set to jmx. Here, specify the port at which the jmx listens for requests from remote hosts.
JMX User, JMX Password, and Confirm Password	These parameters appear only if the Measurement Mode is set to jmx. If JMX requires authentication only (but no security), then ensure that the JMX User and JMX Password parameters are configured with the credentials of a user with read-write access to JMX.

Measurements made by the test
Measurement	Description	Measurement Unit	Interpretation
Data in	Indicates the rate at which data sent from producers is consumed by the broker.	MB/sec
Total produce requests	Indicates produce request rate.	Requests/sec	A very low value is required for this measure.
Messages in	Indicates the rate at which individual messages are consumed by the broker.	Messages/sec
Invalid message CRC records	Indicates the message validation failure rate due to incorrect Cyclic Redundancy Checks (CRC).	Requests/sec	A very low value is required for these measures.
Failed fetch requests	Indicates the fetch request rate for requests that failed.	Requests/sec	A very low value is required for these measures.
Replication data out	Indicates the rate at which data is sent from the broker to other brokers.	MB/sec	This metric is useful to monitor if the broker is a leader for a group of partitions.
Failed produce requests	Indicates the produce request rate for requests that failed.	Requests/sec	A very low value is required for this measure.
Replication data in	Indicates the rate at which data sent from other brokers is consumed by the follower broker.	MB/sec
Invalid offset records	Indicates the message validation failure rate due to non-continuous offset or sequence number in batch.	Records/sec	A very low value is required for this measure.
Data out	Indicates the rate at which data is fetched and read from the broker by consumers.	MB/sec
Fetch message conversions	Indicates the number of fetch message conversions per second for the broker.	Messages/sec
Data rejected	Indicates the aggregate rejected byte rate for the topic.	MB/sec	A very low value is required for these measures.
No key compacted topic records	Indicates the message validation failure rate due to non key specification for compacted topic.	Records/sec	A very low value is required for these measures.
Produce message conversions	Indicates the number of produce message conversions for the broker.	Messages/sec
Reassignment data out	Indicates the outgoing byte rate of reassignment traffic.	MB/sec
Total fetch requests	Indicates the fetch request rate.	Requests/sec
Reassignment data in	Indicates the Incoming byte rate of reassignment traffic.	MB/sec
Invalid magic number records	Indicates the message validation failure rate due to invalid magic number.	Records/sec	A very low value is required for this measure.