AWS Application Load Balancers Test

A listener checks for connection requests, using the protocol and port that you configure. The rules that you define for a listener determine how the load balancer routes requests to its registered targets.

By default, you can configure a maximum of 50 listener for a load balancer. You can however increase this count based on the current and anticipated connection load on the load balancer.

Use the detailed diagnosis to know which listeners are configured on the load balancer, the security policies and actions defined for every listener, and the SSL certificate of each listener.

Each load balancer node periodically sends requests to its registered targets to test their status. These tests are called health checks. These checks are conducted using the health check settings that are configured for the target groups with which a target is registered. After your target is registered, it must pass one health check to be considered healthy.

A non-zero value for this measure means one/more registered targets have passed at least one health check. Ideally therefore, the value of this measure should be high. Use the detailed diagnosis of this measure to identify the healthy targets and the target groups they belong to.

Each load balancer node periodically sends requests to its registered targets to test their status. These tests are called health checks. These checks are conducted using the health check settings that are configured for the target groups with which a target is registered.

After your target is registered, it must pass one health check to be considered healthy. Likewise, if a target fails a configured number of consecutive health checks, then that target will be considered 'Unhealthy'.

A non-zero value for this measure hence means that one/more targets registered with a load balancer are consistently failing health checks. This is a cause for concern. In such a scenario, use the detailed diagnosis of this measure to know which targets are unhealthy, which target groups they belong to, and why their health checks failed. This should help you troubleshoot the failures quickly and effectively.

Here are some possible reasons for health check failures and how to fix them:

• A security group does not allow traffic: The security group associated with an instance must allow traffic from the load balancer using the health check port and health check protocol. You can add a rule to the instance security group to allow all traffic from the load balancer security group. Also, the security group for your load balancer must allow traffic to the instances.

• A network access control list (ACL) does not allow traffic: The network ACL associated with the subnets for your instances must allow inbound traffic on the health check port and outbound traffic on the ephemeral ports (1024-65535). The network ACL associated with the subnets for your load balancer nodes must allow inbound traffic on the ephemeral ports and outbound traffic on the health check and ephemeral ports.

• The ping path does not exist: Create a target page for the health check and specify its path as the ping path.

• Connection time-outs: First, verify that you can connect to the target directly from within the network using the private IP address of the target and the health check protocol. If you can't connect, check whether the instance is over-utilized, and add more targets to your target group if it is too busy to respond. If you can connect, it is possible that the target page is not responding before the health check timeout period. Choose a simpler target page for the health check or adjust the health check settings.

• The target did not return a successful response code: By default, the success code is 200, but you can optionally specify additional success codes when you configure health checks. Confirm the success codes that the load balancer is expecting and that your application is configured to return these codes on success.

• The target response code was malformed or there was an error connecting to the target: Verify that your application responds to the load balancer's health check requests. Some applications require additional configuration to respond to health checks, such as a virtual host configuration to respond to the HTTP host header sent by the load balancer.

This is a good indicator of the connection load on the load balancer.

The value of this measure averaged over a minute is considered for computing the capacity units (LCUs) consumed by the load balancer. These capacity units are in turn used for calculating your hourly Application Load Balancer usage costs.

The value of this measure averaged over a second is considered for computing the capacity units (LCUs) consumed by the load balancer. These capacity units are in turn used for calculating your hourly Application Load Balancer usage costs.

Ideally, the value of this measure should be 0.

This is a good indicator of the workload of the load balancer.

This metric is only incremented for requests where the load balancer node was able to choose a target. Requests that are rejected before a target is chosen are not reflected in this metric.

A non-zero value is desired for this measure.

The value of this measure averaged over a second is considered for computing the capacity units (LCUs) consumed by the load balancer. These capacity units are in turn used for calculating your hourly Application Load Balancer usage costs.

The value of this measure in GB is considered for computing the capacity units (LCUs) consumed by the load balancer. These capacity units are in turn used for calculating your hourly Application Load Balancer usage costs.

Typically, AWS charges you for each hour or partial hour that an Application Load Balancer is running, and the number of Load Balancer Capacity Units (LCU) used per hour.

An LCU measures the dimensions on which the Application Load Balancer processes your traffic (averaged over an hour). The four dimensions measured are:

• New connections: Number of newly established connections per second.

• Active connections: Number of active connections per minute.

• Processed bytes: Number of active connections per minute.

• Rule evaluations: The product of the number of rules processed by your load balancer and the request rate. The first 10 processed rules are free (Rule evaluations = Request rate * (Number of rules processed - 10 free rules).

You are charged only on the dimension with the highest usage. An LCU contains:

• 25 new connections per second

• 3000 active connections per minute

• 1 GB of data processed per hour

• 1000 rule evaluations per second

You may want to keep an eye on the value of this measure to understand your Application load balancer usage. Sudden, significant spikes in this measure value may require investigation. Under such circumstances, its best to track changes to the values of the Active connections, New connections, Rule evaluations. and Processed data by load balancer measures as well. As these measures influence the LCU, knowledge of these measure dynamics may provide pointers to where the major resource spend is.

Use the detailed diagnosis of this measure to know how many requests were received by each target group. This will point you to those target groups with the maximum number of requests.

A consistent increase in the value of this measure is a cause for concern.

Here are some possible causes for poor responsiveness and their probable resolutions:

• The hosts are unhealthy: Verify that all the Application Load Balancer targets are healthy. Refer to the interpretation of the Unhealthy instances measure to know what makes targets 'unhealthy', and how to restore good health.

• The backend instances are overwhelmed by too many requests: Check the sum of the Request count and Active connection count measures of this test for an Application Load Balancer. A sum increase that coincides with the increase in Target response time can indicate that the backend instances are overwhelmed by the request load.

  To resolve this issue, configure an Auto Scaling group for your backend instances.

• There's high CPU utilization on the backend instances: Check the CloudWatch CPUUtilization metric of your backend instances. If CPU utilization is high, or there is a spike in utilization, upgrade your instances to a larger instance type.

• One or more of the targets is faulty: If you're experiencing faulty targets, the follow these steps to resolve the issue:

  • Enable access logging for your load balancer.

  • Download the access logs for the time range when TargetResponseTime is high.

  • Use command line tools to analyze the access logs.

• There are issues with web application dependencies running on backend instances: Run a packet capture on the target to identify the delay in target response. For Linux OS use tcpdump.

Ideally, the value of this measure should be 0. A non-zero value is a cause for concern, as it implies that one/more connections failed owing to TLS errors.

Possible causes include a mismatch of ciphers or protocols or the client failing to verify the server certificate and closing the connection.

Ideally, the value of this measure should be 0. A non-zero value is a cause for concern, as it implies that one/more targets are unreachable.

The primary cause for connection errors is that the target is overloaded and rejecting new connections. If the increased connection errors coincides with an increase in the request rate to a target, then this is the likely culprit.

Another common cause for connection errors is that traffic is being routed to a port specified in the load balancer’s listener that a target is not listening to. This can happen if the process listening on the expected port dies unexpectedly, or if a firewall or security group is not allowing access on the port. If the health check port is different than the listener port, it is possible for this to occur even when the health checks succeed.

Ideally, the value of this measure should be 0. A non-zero value is a cause for concern, as it implies that one/more connections failed owing to TLS errors.

Possible causes include a mismatch of ciphers or protocols or the client failing to verify the server certificate and closing the connection.

Client errors are generated when requests are malformed or incomplete. These requests were not received by the target, other than in the case where the load balancer returns an HTTP 460 error code. This count does not include any response codes generated by the targets.

5xx errors are server errors—meaning the server encountered an issue and is not able to serve the client's request. This count does not include any response codes generated by the targets.

This class of status code indicates that the targets have successfully received, understood, and accepted the requests.

This class of status code indicates the targets must take additional action to complete the request. Many of these status codes are used in URL redirection.

4xx response codes indicate error conditions. Therefore, the value of this measure should ideally be 0.

5xx response codes indicate error conditions. Therefore, the value of this measure should ideally be 0.

The values that this measure can report and the state they represent are detailed in the table below:

If this measure reports a non-zero value, it could imply that one/more targets are in one of the following states:

You can use the detailed diagnosis of this measure to determine the exact state of the targets.

Possible causes for the Lambda user errors include:

• the load balancer did not have permission to invoke the function

• the load balancer received JSON from the function that is malformed or missing required fields

• the size of the request body or response exceeded the maximum size of 1 MB

In case this measure reports a non-zero value, then use the detailed diagnosis of this measure to identify the target groups that experienced the maximum number of Lambda user errors. To know why these errors occurred, check the error_reason field of the access log.

In case this measure reports a non-zero value, then use the detailed diagnosis of this measure to identify the target groups that experienced the maximum number of Lambda internal errors. To know why these errors occurred, check the error_reason field of the access log.