AWS MSK Consumer Test

Kafka consumers are typically part of a consumer group. Consumers read data from brokers. Consumers subscribes to one or more topics and consume published messages by pulling data from the brokers.

This test reports the mean time of the request from the consumer to the broker that waits in the queue and the mean time of the response from the broker to the consumer that waits in the queue. Using this test, administrators can proactively figure out if the mean time for both the request and response queues are too long. The response queue is unbounded and a congested response queue can result in delayed response times and memory pressure on the broker.

Target of the test : AWS Managed Service Kafka

Agent deploying the test : A remote agent

Outputs of the test : One set of results for the target AWS Managed Service Kafka server.

Configurable parameters for the test
Parameter	Description
Test Period	How often should the test be executed.
Host	The IP address of the AWS Managed Service Kafka Broker that is being monitored.
Port	Specify the port number at which the specified HOST listens. By default, this is NULL.
AWS Default Region	This test uses AWS CLI to interact with AWS Managed Service Kafka and pull relevant metrics. To enable the test to connect to AWS, you need to configure the test with the name of the region to which all requests for metrics should be routed, by default. Specify the name of this AWS Default Region, here.
AWS Access Key ID, AWS Secret Access Key and Confirm Password	To monitor AWS Managed Service Kafka, the eG agent has to be configured with the access key and secret key of a user with a valid AWS account. For this purpose, we recommend that you create a special user on the AWS cloud, obtain the access and secret keys of this user, and configure this test with these keys. The procedure for this has been detailed in the Obtaining an Access key and Secret key topic. Make sure you reconfirm the access and secret keys you provide here by retyping it in the corresponding Confirm Password text box.
Timeout Seconds	Specify the maximum duration (in seconds) for which the test will wait for a response from the server. The default is 10 seconds.

Measurements made by the test
Measurement	Description	Measurement Unit
Consumer local time	Indicates the mean time that the consumer request is processed at the leader.	Seconds
Consumer request queue time	Indicates the mean time that the consumer request waits in the request queue.	Seconds
Consumer response queue time	Indicates the mean time that the consumer request waits in the response queue.	Seconds
Consumer response send time	Indicates the mean time for the consumer to send a response.	Seconds
Consumer total time	Indicates the mean total time that consumers spend on fetching data from the broker.	Seconds