AWS CloudWatch Logs Test

AWS CloudWatch Logs is a highly available, scalable, durable, and secure service to manage your operating system and application log files. It allows you to ingest, store, filter, search, and archive the logs, reducing your operational burden and allowing you to focus on your application and your business.

Administrators typically reach for logs whenever their applications and systems on the cloud encounter issues. This is because, logs greatly help problem diagnosis and troubleshooting. In fact, if administrators need to perform any custom processing on the logs for deep-dive diagnostics or if they want to load these logs on to other systems for deeper analytics, they even configure these logs to be automatically delivered to other services such as Amazon Kinesis stream, Amazon Kinesis Data Firehose stream, or AWS Lambda. In such situations, if delivery errors occur, logs may not be able to reach the destination services in time. This in turn can impede analysis and delay the administrator's troubleshooting efforts. As a result, the downtime of critical applications on the cloud will increase! To avoid this, it is good practice to frequently run the AWS CloudWatch Logs test!

This test automatically discovers the log groups that have been configured on AWS Cloud. A log group can have one or multiple log streams in it. Each of these streams will share the same retention policy, monitoring setting or access control permissions. For each log group, the test tracks the log events and log data that is forwarded by each group to AWS service destinations such as Amazon Kinesis stream, Amazon Kinesis Data Firehose stream, or AWS Lambda. In the process, the test promptly captures and reports delivery errors and also brings to light instances where delivery has been throttled. This way, the test reveals bottlenecks in the delivery of logs to AWS services, pinpoints the log groups experiencing the bottlenecks, and thus hastens appropriate corrective action.

Optionally, you can configure this test to report metrics for each log destination or for every subscription filter. This enables administrators to quickly and easily understand if specific destinations / filters are problem-prone.

Target of the test: Amazon Cloud

Agent deploying the test : A remote agent

Outputs of the test : One set of results for each log group / destination / subscription filter

First-level descriptor: AWS Region

Second-level descriptor: Log group / destination / subscription filter, depending upon the option chosen from the Logs Filter Name parameter of this test

Configurable parameters for the test
Parameter Description

Test Period

How often should the test be executed.

Host

The host for which the test is to be configured.

Access Type

eG Enterprise monitors the AWS cloud using AWS API. By default, the eG agent accesses the AWS API using a valid AWS account ID, which is assigned a special role that is specifically created for monitoring purposes. Accordingly, the Access Type parameter is set to Role by default. Furthermore, to enable the eG agent to use this default access approach, you will have to configure the eG tests with a valid AWS Account ID to Monitor and the special AWS Role Name you created for monitoring purposes.

Some AWS cloud environments however, may not support the role-based approach. Instead, they may allow cloud API requests only if such requests are signed by a valid Access Key and Secret Key. When monitoring such a cloud environment therefore, you should change the Access Type to Secret. Then, you should configure the eG tests with a valid AWS Access Key and AWS Secret Key.

Note that the Secret option may not be ideal when monitoring high-security cloud environments. This is because, such environments may issue a security mandate, which would require administrators to change the Access Key and Secret Key, often. Because of the dynamicity of the key-based approach, Amazon recommends the Role-based approach for accessing the AWS API.

AWS Account ID to Monitor

This parameter appears only when the Access Type parameter is set to Role. Specify the AWS Account ID that the eG agent should use for connecting and making requests to the AWS API. To determine your AWS Account ID, follow the steps below:

  • Login to the AWS management console. with your credentials.

  • Click on your IAM user/role on the top right corner of the AWS Console. You will see a drop-down menu containing the Account ID (see Figure 1).

    Figure 1 : Identifying the AWS Account ID

AWS Role Name

This parameter appears when the Access Type parameter is set to Role. Specify the name of the role that you have specifically created on the AWS cloud for monitoring purposes. The eG agent uses this role and the configured Account ID to connect to the AWS Cloud and pull the required metrics. To know how to create such a role, refer to Creating a New Role.

AWS Access Key, AWS Secret Key, Confirm AWS Access Key, Confirm AWS Secret Key

These parameters appear only when the Access Type parameter is set to Secret.To monitor an Amazon cloud instance using the Secret approach, the eG agent has to be configured with the access key and secret key of a user with a valid AWS account. For this purpose, we recommend that you create a special user on the AWS cloud, obtain the access and secret keys of this user, and configure this test with these keys. The procedure for this has been detailed in the Obtaining an Access key and Secret key topic. Make sure you reconfirm the access and secret keys you provide here by retyping it in the corresponding Confirm text boxes.

Proxy Host and Proxy Port

In some environments, all communication with the AWS cloud and its regions could be routed through a proxy server. In such environments, you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics. To enable metrics collection via a proxy, specify the IP address of the proxy server and the port at which the server listens against the Proxy Host and Proxy Port parameters. By default, these parameters are set to none , indicating that the eG agent is not configured to communicate via a proxy, by default.

Proxy User Name, Proxy Password, and Confirm Password

If the proxy server requires authentication, then, specify a valid proxy user name and password in the Proxy User Name and Proxy Password parameters, respectively. Then, confirm the password by retyping it in the Confirm Password text box. By default, these parameters are set to none, indicating that the proxy sever does not require authentication by default.

Proxy Domain and Proxy Workstation

If a Windows NTLM proxy is to be configured for use, then additionally, you will have to configure the Windows domain name and the Windows workstation name required for the same against the Proxy Domain and Proxy Workstation parameters. If the environment does not support a Windows NTLM proxy, set these parameters to none.

Exclude Region

Here, you can provide a comma-separated list of region names or patterns of region names that you do not want to monitor. For instance, to exclude regions with names that contain 'east' and 'west' from monitoring, your specification should be: *east*,*west*

Logs Filter Name

By default, this parameter is set to LogGroupName. This means that by default, this test will report metrics for log group. Log groups define groups of log streams that share the same retention, monitoring, and access control settings. Each log stream has to belong to one log group. A log stream is a sequence of log events that share the same source. For example, a log stream may be associated with an Apache access log on a specific host.

If required, you can override this default setting by setting the Logs Filter Name parameter to one of the following:

  • FilterName - If you want this test to report metrics for every subscription filter, select the FilterName option from the Logs Filter Name drop-down list.

    Subscriptions are used to get access to a real-time feed of log events from CloudWatch Logs and have it delivered to other services such as an Amazon Kinesis stream, Amazon Kinesis Data Firehose stream, or AWS Lambda for custom processing, analysis, or loading to other systems. To begin subscribing to events, you need to create subscription filters. A subscription filter defines the filter pattern to use for filtering which log events get delivered to your AWS resource, as well as information about where to send matching log events to.

  • DestinationType - If you want this test to report metrics for every destination for your log events (eg., Amazon Kinesis stream, Amazon Kinesis Data Firehose stream, or AWS Lambda), select the DestinationType option from the Logs Filter Name drop-down list.
Measurements made by the test
Measurement Description Measurement Unit Interpretation

Incoming data

Indicates the volume of log events in uncompressed data uploaded to this log group.

KB

This measure is reported only if the 'Logs Filter Name' flag is set to 'LogGroupName'.

Incoming log events

Indicates the number of log events uploaded to this log group.

Number

This measure is reported only if the 'Logs Filter Name' flag is set to 'LogGroupName'.

Forwarded data

By default, this measure represents the volume of log events in uncompressed data that is forwarded from this log group to one/more AWS resource destinations.

If the Logs Filter Name is set to FilterName, then this measure represents the amount of log data that is forwarded via this subscription filter to the resource destinations defined within that filter.

If the Logs Filter Name is set to DestinationType, then this measure represents the amount of log data that is forwarded to this AWS resource destination via one/more subscription filters.

KB

Forwarded log events

By default, this measure represents the number of log events forwarded from this log group to one/more AWS resource destinations.

If the Logs Filter Name is set to FilterName, then this measure represents the number of log events forwarded via this subscription filter to the resource destinations defined within that filter.

If the Logs Filter Name is set to DestinationType, then this measure represents the number of log events forwarded to this AWS resource destination via one/more subscription filters.

Number

Delivery errors

By default, this measure represents the number of log events in this log group that encountered errors when they were being forwarded to one/more AWS resource destinations.

If the Logs Filter Name is set to FilterName, then this measure represents the number of log events that encountered errors when being forwarded via this subscription filter to the resource destinations defined within that filter.

If the Logs Filter Name is set to DestinationType, then this measure represents the number of log events that encountered errors when they were being forwarded to this AWS resource destination via one/more subscription filters.

Number

Ideally, the value of this measure should be 0.

If the count of errors consistently increase for log events in a specific log group, or to a specific destination, or via a specific subscription filter, then that log group / filter / destination should be taken up for closer scrutiny.

Delivery throttling

By default, the number of log events in this log group that were throttled when being forwarded to one/more resource destinations.

If the Logs Filter Name is set to FilterName, then this measure represents the number of log events that were throttled when being forwarded via this subscription filter to the resource destinations defined within that filter.

If the Logs Filter Name is set to DestinationType, then this measure represents the number of log events that were throttled when they were being forwarded to this AWS resource destination via one/more subscription filters.

Number

Ideally, the value of this measure should be low.