{"id":18703,"date":"2022-03-15T12:23:41","date_gmt":"2022-03-15T16:23:41","guid":{"rendered":"https:\/\/www.eginnovations.com\/blog\/?p=18703"},"modified":"2025-07-04T05:00:55","modified_gmt":"2025-07-04T09:00:55","slug":"aws-cloudwatch-monitoring-2","status":"publish","type":"post","link":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/","title":{"rendered":"Using an AWS CloudWatch Agent for System Monitoring of AWS EC2 Cloud Instances"},"content":{"rendered":"<div class=\"inner_content\">\n<p>In this blog post, I follow on from my previous blog on <a href=\"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/\" target=\"blank\" rel=\"noopener noreferrer\">AWS CloudWatch Part 1 of 2<\/a> to explore how you can go beyond basic agentless AWS CloudWatch monitoring and get detailed insights into EC2 instances by deploying the CloudWatch agent. I&#8217;ll also the key information and planning you need to do this. I\u2019ll also cover how eG Enterprise offers out-of-the-box functionality to avoid complex JSON scripting or tooling to implement monitoring.<\/p>\n<p><em>Updated October 2023:<\/em> To add links to useful information on known CloudWatch limitations: <a href=\"https:\/\/www.techtarget.com\/searchcloudcomputing\/tip\/Be-aware-of-these-CloudWatch-Logs-limits-and-quotas\">Be aware of these CloudWatch Logs limits and quotas | TechTarget<\/a><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Installing_a_CloudWatch_Agent_for_Detailed_Monitoring_of_AWS_EC2_Instances\"><\/span>Installing a CloudWatch Agent for Detailed Monitoring of AWS EC2 Instances<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>If you are deploying Linux or Windows OSs on EC2, you will probably want to deploy the CloudWatch agent on each instance to enable a wider range of in-guest metrics. If you install the agent on an Amazon EC2 instance, the metrics it collects are in addition to the agentless metrics enabled by default for Amazon EC2 instances. For an overview of Agentless vs Agent-based monitoring, please see: <a href=\"https:\/\/www.eginnovations.com\/blog\/agentless-vs-agent-based-monitoring\/\" target=\"blank\" rel=\"noopener noreferrer\">Agentless Monitoring vs. Agent Based.<\/a><\/p>\n<p>There is a reasonable <a href=\"https:\/\/aws.amazon.com\/cloudwatch\/getting-started\/\" target=\"blank\" rel=\"noopener noreferrer\">\u201cGetting Started Guide\u201d<\/a> for the CloudWatch agent and it is worth reviewing the metrics the agent will enable on <a href=\"https:\/\/docs.aws.amazon.com\/AmazonCloudWatch\/latest\/monitoring\/metrics-collected-by-CloudWatch-agent.html\" target=\"blank\" rel=\"noopener noreferrer\">Windows Server instances<\/a> and also upon <a href=\"https:\/\/docs.aws.amazon.com\/AmazonCloudWatch\/latest\/monitoring\/metrics-collected-by-CloudWatch-agent.html#linux-metrics-enabled-by-CloudWatch-agent\" target=\"blank\" rel=\"noopener noreferrer\">Linux\/MacOS instances.<\/a> On a server running Microsoft Windows OS, installing the CloudWatch agent enables you to collect the metrics associated with any of the counters in Windows Performance Monitor. The CloudWatch metric names for these counters are created by putting a space between the object name and the counter name. For example, the % Interrupt Time counter of the Processor object is given the metric name \u201cProcessor % Interrupt Time\u201d in CloudWatch. For more information about Windows Performance Monitor counters, see the <a href=\"https:\/\/docs.microsoft.com\/en-us\/windows-server\/remote\/remote-desktop-services\/rds-rdsh-performance-counters\" target=\"blank\" rel=\"noopener noreferrer\">Microsoft Windows Server documentation.<\/a><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-21874 size-full\" src=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/03\/AWS-CloudWatch-agent.jpg?20\" alt=\"What AWS CloudWatch monitoring offers\" width=\"500\" height=\"471\" border=\"0\" \/><\/p>\n<div class=\"img_caption\">Figure 1: How an AWS CloudWatch agent complements basic metrics regarding EC2 instance health<\/div>\n<p>The steps involved in deploying CloudWatch\u2019s <a href=\"https:\/\/www.eginnovations.com\/blog\/agentless-vs-agent-based-monitoring\/\" target=\"blank\" rel=\"noopener noreferrer\">agented<\/a> monitoring have evolved but there is a fair amount of configuration of JSON files or python required even to define which perfmon counters you wish to collect.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Ease_of_Agent_Deployment_and_Configuring_Required_Parameters_on_AWS_CloudWatch\"><\/span>Ease of Agent Deployment and Configuring Required Parameters on AWS CloudWatch<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"margin-bottom: 15px;\">There is a degree of complexity involved in configuring an AWS CloudWatch agent. Manipulating the JSON configuration files is cumbersome. You need to:<\/p>\n<ul>\n<li>Which qualifying metrics to monitor<\/li>\n<li>Same metrics have different name and syntax across OS (Win, Linux, AIX, etc.)<\/li>\n<li>Maintain and version control the config.json file<\/li>\n<li>Handle lengthy Config files \u2013 With each increasing metric, the config file tend to become long and prone to mistakes.<\/li>\n<\/ul>\n<p style=\"margin-bottom: 15px;\">For example, to monitor processes on a system:<\/p>\n<ul>\n<li>Process name \u2013 Each process name must be defined<\/li>\n<li>Metric name \u2013 Each metric for the process must be defined<\/li>\n<li>Metric Collection Interval \u2013 Each metric section for metric Collection Interval must be defined if it is to be set other than 60 seconds<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-19834 size-full\" src=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/CW1.jpg\" alt=\"Configuring AWS EC2 detailed metrics in AWS CloudWatch\" width=\"750\" height=\"500\" border=\"0\" srcset=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/CW1.jpg 750w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/CW1-300x200.jpg 300w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/CW1-310x207.jpg 310w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/CW1-140x93.jpg 140w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/><\/p>\n<div class=\"img_caption\">Figure 2: AWS config.json file for configuring process monitoring<\/div>\n<h2><span class=\"ez-toc-section\" id=\"Implementing_Alerting_Reporting_and_Dashboards\"><\/span>Implementing Alerting, Reporting and Dashboards<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>There are lots of tools within AWS CloudWatch to enable you to configure a monitoring solution but very little ready-to-go functionality. If you are relying on native AWS tools, you may find it helpful to familiarize yourself with the numerous repositories of tutorials that exist to cover common requirements, such as Jackson Connell\u2019s <a href=\"https:\/\/www.threatstack.com\/blog\/50-best-aws-cloudwatch-tutorials\" target=\"blank\" rel=\"noopener noreferrer\">\u201c50 Best AWS CloudWatch Tutorials\u201d.<\/a><\/p>\n<h4><span class=\"ez-toc-section\" id=\"Out-of-the-box_Dashboard\"><\/span>Out-of-the-box Dashboard<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>Out-of-the-box there are basic metric dashboards available, but the observability of configured parameters is a somewhat cumbersome task for administrators as they have to toggle between multiple screens and pages to monitor\/eyeball the metrics. The basic view of the Default Metrics can be viewed under \u2013 EC2 Instances -&gt; Monitoring (see below):<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-19817 size-full\" src=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-screen.jpg\" alt=\"AWS CloudWatch monitoring screen\" width=\"750\" height=\"292\" border=\"0\" srcset=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-screen.jpg 750w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-screen-300x117.jpg 300w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-screen-310x121.jpg 310w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-screen-140x55.jpg 140w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/><\/p>\n<div class=\"img_caption\">Figure 3: CloudWatch monitoring screen<\/div>\n<p>The minimal dashboard functionality mean that many AWS users export the CloudWatch metrics stored in logs to other BI tools, such as ELK or Grafana which offer more advanced visualization and there are charges applicable for these tools.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Configuring_Alert_Thresholds_and_Alarms\"><\/span>Configuring Alert Thresholds and Alarms<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The majority of administrators, especially with large deployments will want to configure their monitoring systems to proactively notify and alert them of problems. It is unviable for an administrator to physically watch dashboards all day or attempt to manually\/visually correlate metrics of concern indicative of an issue. AWS does allow thresholds and alarms to be configured but this is a manual process requiring an admin to decide on the thresholds at which individual metrics should trigger alerts; some problems by their very nature will trigger alert storms.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-19821 alignright\" src=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-conditions.jpg\" alt=\"CloudWatch monitoring conditions\" width=\"480\" height=\"266\" border=\"0\" srcset=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-conditions.jpg 480w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-conditions-300x166.jpg 300w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-conditions-310x172.jpg 310w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/cloudwatch-monitoring-conditions-140x78.jpg 140w\" sizes=\"auto, (max-width: 480px) 100vw, 480px\" \/><\/p>\n<p>The conditions based on which alerts can be triggered allow static and \u201cAnomaly detection\u201d thresholds (comparable to \u201cDynamic\u201d thresholds in Microsoft Azure cloud) to be set. <a href=\"https:\/\/searchitoperations.techtarget.com\/tip\/Monitoring-thresholds-determine-IT-performance-alerts\" target=\"blank\" rel=\"noopener noreferrer\">The limitations of these two types of thresholds are well known,<\/a> and in practice, <a href=\"https:\/\/www.eginnovations.com\/product\/it-infrastructure-monitoring\" target=\"blank\" rel=\"noopener noreferrer\">enterprise-class monitoring solutions<\/a>, such as eG Enterprise, have integrated functionality to combine the two genres to reduce false positives; as of today, cloud native monitoring within AWS and Azure does not allow this.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Alert_and_Event_Correlation_and_Root-Cause_Analysis\"><\/span>Alert and Event Correlation and Root-Cause Analysis<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In practice, a single IT issue can trigger numerous thresholds to be crossed generating alerts, where most alarms raised are secondary symptoms of the true fault or bottleneck. The user-implemented thresholding and alerting in cloud native tools as of the time of writing this blog post do not provide the <a href=\"https:\/\/www.eginnovations.com\/blog\/aiops-tools-capabilities\/\" target=\"blank\" rel=\"noopener noreferrer\">correlation, event filtering, and AIOps root cause diagnosis capabilities<\/a> available in dedicated enterprise monitoring products.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"LicensingCosts_of_AWS_CloudWatch_for_Detailed_Monitoring_of_EC2\"><\/span>Licensing\/Costs of AWS CloudWatch for Detailed Monitoring of EC2<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"margin-bottom: 15px;\">There are a large number of parameters that affect <a href=\"https:\/\/aws.amazon.com\/cloudwatch\/pricing\/\" target=\"blank\" rel=\"noopener noreferrer\">the costs of CloudWatch<\/a> and it can become extremely complex even with basic monitoring. Many users underestimate the costs as multiple cost factors must be considered for even single service or metric. Key considerations and factors you need to consider include:<\/p>\n<ul>\n<li>Configuration and coverage:\n<ul>\n<li>For CPU utilization &#8211; Different config.json files must be maintained for same CPU utilization objective cross different OSs.<\/li>\n<li>Process-wise CPU utilization &#8211; Each process must be distinctly added to config.json file.<\/li>\n<li>If your eventual goal was to be able to routinely monitor the \u201cTop 10 CPU consuming Processes\u201d (a standard feature in monitoring products) \u2013 it is essentially impossible to add each process to config.json file. Even if added, the metric counts will run into 1000s.<\/li>\n<\/ul>\n<\/li>\n<li>Resolution-based licensing:\n<ul>\n<li>Multiple resolutions are available: (&lt;1min, &gt;1min, and 5 mins) \u2013 there is a different costing for different time resolutions.<\/li>\n<\/ul>\n<\/li>\n<li>Data retention:\n<ul>\n<li>Data points with a period of 60 seconds (1 minute) are available for 15 days.<\/li>\n<li>Data points with a period of 300 seconds (5 minute) are available for 63 days.<\/li>\n<li>Data points with a period of 3600 seconds (1 hour) are available for 455 days (15 months).<\/li>\n<\/ul>\n<\/li>\n<li>Dashboard views \u2013 CloudWatch implements per dashboard view based costing.<\/li>\n<li>Log Monitoring:\n<ul>\n<li>Data volume \u2013 per GB of data file transferred<\/li>\n<li>Storage \u2013 per GB of data stored on S3<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>My colleague wrote a <a href=\"https:\/\/www.eginnovations.com\/blog\/reduce-azure-monitoring-cost\/\" target=\"blank\" rel=\"noopener noreferrer\">recent blog post covering estimating data volumes when deploying Azure Log Analytics and Azure Monitor<\/a> to monitor WVD\/AVD (Windows\/Azure Virtual Desktops) and the process of estimating data volumes is similar on AWS. Particularly when monitoring application processes, it is likely that you will need to accumulate some real-world data of your AWS deployment in production to gain realistic estimates of volumes and costs on-going.<\/p>\n<p>AWS has done a respectable job of documenting the process of estimating costs via a number of sample calculations; if your deployment closely matches one of these examples (found <a href=\"https:\/\/aws.amazon.com\/cloudwatch\/pricing\/\" target=\"blank\" rel=\"noopener noreferrer\">at the bottom of this page<\/a>), they may be very helpful. They give a good overview of many additional types of monitoring that can be added to a deployment and customizations and configurations an administrator is likely needed to make.<\/p>\n<div class=\"link_list_style\" style=\"margin: 25px auto 20px; padding: 25px 20px 10px;\">\n<h4><span class=\"ez-toc-section\" id=\"Example_1_-_EC2_Detailed_Monitoring\"><\/span>Example 1 \u2013 EC2 Detailed Monitoring<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>The number of metrics sent by your EC2 instance as detailed monitoring is dependent on the EC2 instance type &#8211; see <a href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/viewing_metrics_with_cloudwatch.html#ec2-cloudwatch-metrics\" target=\"blank\" rel=\"noopener noreferrer\">Instance Metrics documentation<\/a> for details. This example assumes 7 metrics, which covers the most commonly used instance types.<\/p>\n<p>If your application runs on 10 Amazon EC2 instances 24&#215;7 for a 30-day month, and you enable EC2 Detailed Monitoring on all instances, your charges would be as follows:<\/p>\n<p>Total number of metrics = 7 metrics per instance * 10 instances = 70 metrics<\/p>\n<p>Monthly CloudWatch Metrics Charges @$0.30 per custom metric = 70 * $0.30 = $21<\/p>\n<p>Monthly CloudWatch charges = $21 per month<\/p>\n<p>Once you exceed 10,000 total metrics, then volume pricing tiers will apply \u2013 see <a href=\"https:\/\/aws.amazon.com\/cloudwatch\/pricing\/\" target=\"blank\" rel=\"noopener noreferrer\">metrics pricing table<\/a> for details.<\/p>\n<\/div>\n<h2><span class=\"ez-toc-section\" id=\"Summary_of_the_Challenges_of_Using_AWS_CloudWatch_for_Detailed_Monitoring\"><\/span>Summary of the Challenges of Using AWS CloudWatch for Detailed Monitoring<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"margin-bottom: 15px;\">If using AWS CloudWatch alone, an administrator should consider and plan how to invest time to address inherent challenges, including:<\/p>\n<ul>\n<li><strong>Monitoring \u2013<\/strong> Though you can monitor at very granular levels, it will become very complex to manage the config file defining what metrics to collects. Manual maintenance of config files is prone to errors.<\/li>\n<li><strong>Event correlation \u2013<\/strong> A lot of effort and skills will be required to correlate events, suppress false positive, and help drill down to probable cause.<\/li>\n<li><strong>Licensing and Costs &#8211;<\/strong> This will remain an open-ended question. Budgeting will be difficult as you grow and may lead to unknown expenses and leakages. Additionally, one has to keep in mind the subsequent services like SNS, SSM, EBS, etc. that will be required for effective monitoring and reporting.<\/li>\n<li><strong>Ease of deployment \u2013<\/strong> AWS Systems Manager may help in a few of the monitoring operations like config release management. However, config files still remain a manual and complicated task. Because of this, and limitations of AWS in consuming a number of metrics, many admins create their own scripts, do the processing needed in their script, and just use AWS as the display tool.<\/li>\n<li><strong>Observability \u2013<\/strong> Monitoring just by looking at an alarm is not an effective method. Maneuvering across various dependent components and drilling down for root-cause analysis is a key to faster mean time to repair (MTTR).<\/li>\n<li><strong>Dashboards \u2013<\/strong> They can be extremely handy but will require thoughtful and methodical mechanics in the prior steps of qualifying metrics and building strong correlations. Also, scripting and programming knowledge on LAMBDA, Python, etc. plays a major role in building an effective dashboard for respective teams.<\/li>\n<\/ul>\n<h2><span class=\"ez-toc-section\" id=\"How_eG_Enterprise_Simplifies_Detailed_Monitoring_of_AWS_EC2_Instances\"><\/span>How eG Enterprise Simplifies Detailed Monitoring of AWS EC2 Instances<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>For many of our customers, especially those choosing hybrid deployments, choosing the native AWS CloudWatch options and an AWS-only interface doesn\u2019t make sense as it is an extra monitoring tool requiring staff to train and maintain and monitor costs. AWS themselves acknowledge that for many of their customers, CloudWatch alone is insufficient or simply too complex for some customers.<\/p>\n<p><a href=\"https:\/\/www.eginnovations.com\/product\" target=\"blank\" rel=\"noopener noreferrer\">eG Enterprise<\/a> makes it very simple to monitor your AWS EC2 instances. By integrating with AWS CloudWatch APIs, eG Enterprise provides an \u201coutside view\u201d of the performance of all the EC2 instances. Other than providing credentials to access AWS CloudWatch APIs, there is no configuration necessary. Pre-configured thresholds alert administrators to issues that affect the performance of EC2 instances and applications running on them. Figure 4 below shows some of the key metrics for EC2 instance performance on eG Enterprise.<\/p>\n<p><a href=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/monitoring-aws-ec2-instances-view.png\" data-rel=\"lightbox-image-0\" data-rl_title=\"\" data-rl_caption=\"\" title=\"\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-19822 size-full\" src=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/monitoring-aws-ec2-instances.jpg\" alt=\"Monitoring AWS EC2 instances with eG Enterprise\" width=\"750\" height=\"420\" border=\"0\" srcset=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/monitoring-aws-ec2-instances.jpg 750w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/monitoring-aws-ec2-instances-300x168.jpg 300w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/monitoring-aws-ec2-instances-310x174.jpg 310w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/monitoring-aws-ec2-instances-140x78.jpg 140w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/><\/a><\/p>\n<div class=\"img_caption\">Figure 4: Monitoring of AWS EC2 instances in eG Enterprise<\/div>\n<p>In-guest metrics can be obtained by simply installing an agent inside the EC2 instance. You need not manually install the agent each time. You can embed the agent in your Amazon Machine Images (AMI), or you can use an orchestration tool like Terraform or Cloud Formation to install the agents as your EC2 instances are provisioned. The default built-in profile of a Windows or Linux system on eG Enterprise ensures that admins don\u2019t have to spend time and effort choosing what metrics to collect. Automatically, the agents collect the pre-defined metrics and you can visualize them in the eG monitoring dashboard (see Figure 5). Everything from CPU and memory utilization, disk IOPS, page file usage, network traffic and errors, TCP retransmissions, status of key services and events in the Windows event logs are monitored by default.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-19823 size-full\" src=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/ec2-microsoft-windows.jpg\" alt=\"Monitoring of an EC2 instance running Microsoft Windows OS\" width=\"750\" height=\"419\" border=\"0\" srcset=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/ec2-microsoft-windows.jpg 750w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/ec2-microsoft-windows-300x168.jpg 300w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/ec2-microsoft-windows-310x173.jpg 310w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/ec2-microsoft-windows-140x78.jpg 140w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/><\/p>\n<div class=\"img_caption\">Figure 5: Stack model of an EC2 instance running Microsoft Windows OS<\/div>\n<p>Different visualizations are available. Besides the layer model view, there are other dashboards you can use to visualize the data. If you want, you can create your own custom dashboard as well.<\/p>\n<p>Detailed diagnosis is also automated in eG Enterprise. When a metric changes state to an abnormal state, the agent automatically collects additional details to enable IT admins troubleshoot the problem quickly. For example, in Figure 6 below, you see the detailed diagnosis of a Linux instance that is seeing high CPU utilization. The detailed diagnosis shows the top 10 CPU consuming processes on this instance. Again, administrators do not have to pick and choose what they want to see. eG Enterprise\u2019s detailed diagnosis is pre-tuned to highlight the key details needed to troubleshoot each threshold violation.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-19824\" src=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/linux-ec2.jpg\" alt=\"Detailed diagnosis of a Linux EC2 instance\" width=\"750\" height=\"232\" border=\"0\" srcset=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/linux-ec2.jpg 750w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/linux-ec2-300x93.jpg 300w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/linux-ec2-310x96.jpg 310w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/linux-ec2-140x43.jpg 140w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/><\/p>\n<div class=\"img_caption\">Figure 6: Detailed diagnosis of a Linux EC2 instance showing the top processes taking up CPU<\/div>\n<p><a href=\"https:\/\/www.eginnovations.com\/product\/application-performance-monitoring\/free-trial\" rel=\"noopener noreferrer\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-21057 size-full\" src=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/01\/single-freetrial-banner.jpg\" alt=\"Single pane of glass monitoring with eG Enterprise\" width=\"850\" height=\"195\" border=\"0\" srcset=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/01\/single-freetrial-banner.jpg 850w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/01\/single-freetrial-banner-300x69.jpg 300w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/01\/single-freetrial-banner-768x176.jpg 768w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/01\/single-freetrial-banner-800x184.jpg 800w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/01\/single-freetrial-banner-310x71.jpg 310w, https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/01\/single-freetrial-banner-140x32.jpg 140w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/a><\/p>\n<p>Here\u2019s a comparison of the capabilities of eG Enterprise and AWS CloudWatch as it relates to monitoring EC2 instances.<\/p>\n<table class=\"table_design numbers\" style=\"width: 100%;\">\n<tbody>\n<tr class=\"head_table\">\n<td><\/td>\n<td><strong>AWS CloudWatch<\/strong><\/td>\n<td><strong>eG Enterprise<\/strong><\/td>\n<\/tr>\n<tr>\n<td>Ability to collect key performance metrics per instance including CPU credit balance<\/td>\n<td>Yes<\/td>\n<td>Yes, by integration with CloudWatch APIs<\/td>\n<\/tr>\n<tr>\n<td>Simple agent deployment for in-guest metrics<\/td>\n<td>Yes<\/td>\n<td>Yes<\/td>\n<\/tr>\n<tr>\n<td>Ability to collect system KPIs within the guest without much configuration<\/td>\n<td>No<\/td>\n<td>Yes<\/td>\n<\/tr>\n<tr>\n<td>Ability to collect detailed diagnosis data for rapid troubleshooting<\/td>\n<td>No<\/td>\n<td>Yes<\/td>\n<\/tr>\n<tr>\n<td>Cost depends on metric collection frequency and number of metrics collected<\/td>\n<td>Yes<\/td>\n<td>No<\/td>\n<\/tr>\n<tr>\n<td>Extend the same agent to collect application-level performance metrics<\/td>\n<td>No<\/td>\n<td>Yes<\/td>\n<\/tr>\n<tr>\n<td>Ability to collect configuration data about the OS (patches, hot fixes, etc.) and report on changes<\/td>\n<td>No<\/td>\n<td>Yes<\/td>\n<\/tr>\n<tr>\n<td>Ability to automatically monitor key system logs<\/td>\n<td>Only by writing custom scripts<\/td>\n<td>Out of the box<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>More details of AWS monitoring with eG Enterprise is available here: <a href=\"https:\/\/www.eginnovations.com\/supported-technologies\/aws-monitoring\" target=\"blank\" rel=\"noopener noreferrer\">eG Enterprise for AWS Monitoring<\/a>. eG Enterprise is also available as SaaS in multiple AWS regions from AWS marketplace, here: <a href=\"https:\/\/aws.amazon.com\/marketplace\/pp\/prodview-pn745igznyfre\" target=\"blank\" rel=\"noopener noreferrer\">AWS Marketplace: eG Enterprise Express Cloud (amazon.com).<\/a><\/p>\n<p style=\"margin-bottom: 15px;\">Using eG Enterprise allows AWS users to go beyond CloudWatch to:<\/p>\n<ul>\n<li>Monitor all their AWS components such as EKS, EC2, ELB, RDS, SNS, Amazon Lambda and more, within a single product as a \u201csingle pane of glass\u201d, alongside other clouds, such as Google GCP or Microsoft Azure and alongside on-premises physical and virtualized infrastructure, such as Citrix\/VMware\/Nutanix.<\/li>\n<li>Use detailed out-of-the-box or customizable reports for forecasting, capacity planning, and management insights based on data that can be retained on whatever timescales suit an organization.<\/li>\n<li>Deploy metric collection and set up alert\/alarm thresholds out-of-the-box with pre-configured industry standard values that can be modified via a GUI, avoiding the need for administrators to code or configure with python or JSON files.<\/li>\n<li><em>Update October 2023:<\/em> Since writing this article I have found a useful article that details some known CloudWatch limitations with which eG Enterprise helps, see: <a href=\"https:\/\/www.techtarget.com\/searchcloudcomputing\/tip\/Be-aware-of-these-CloudWatch-Logs-limits-and-quotas\">Be aware of these CloudWatch Logs limits and quotas | TechTarget<\/a>.<\/li>\n<\/ul>\n<h3><span class=\"ez-toc-section\" id=\"Learn_More\"><\/span>Learn More<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul>\n<li>AWS Performance Monitoring with eG Enterprise \u2013 <a href=\"https:\/\/www.eginnovations.com\/supported-technologies\/aws-monitoring\" target=\"blank\" rel=\"noopener noreferrer\">An overview of key capabilities and features<\/a><\/li>\n<li>Read about Synthetic Monitoring and Testing for Amazon WorkSpaces: <a href=\"https:\/\/www.eginnovations.com\/blog\/synthetic-monitoring-amazon-workspaces\/\" target=\"blank\" rel=\"noopener noreferrer\">Synthetic Monitoring of Amazon Workspaces | eG Innovations<\/a><\/li>\n<li>Read how Israel\u2019s largest supermarket chain leverages an eG Enterprise integration with CloudWatch to manage large and complex auto-scaling deployments: <a href=\"https:\/\/www.eginnovations.com\/case-studies\/cloud-issues-problems-management\" target=\"blank\" rel=\"noopener noreferrer\">Cloud Issues &amp; Problems &#8211; Management Case Study | eG Innovations<\/a><\/li>\n<li>A real-life deep-dive post-mortem case study \u2013 debugging slow performance on AWS public cloud burstable instances on EC2, see: <a href=\"https:\/\/www.eginnovations.com\/blog\/aws-ec2-monitoring-tools\/\" target=\"blank\" rel=\"noopener noreferrer\">AWS EC2 Monitoring Tools | eG Innovations<\/a><\/li>\n<li>More on how eG Enterprise leverages AIOps technologies for event correlation, anomaly detection and root-cause diagnostic analysis: <a href=\"https:\/\/www.eginnovations.com\/blog\/aiops-tools-capabilities\/\" target=\"blank\" rel=\"noopener noreferrer\">AIOps Tools \u2013 8 Proactive Monitoring Tips<\/a><\/li>\n<li>Cloud Migration Strategy: A Framework for Cloud Adoption and Target Cloud Models &#8211; <a href=\"https:\/\/www.eginnovations.com\/blog\/6-rs-aws-migration-strategies\/\" target=\"blank\" rel=\"noopener noreferrer\">Define your Pathway to Cloud Migration<\/a><\/li>\n<\/ul>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>In this blog post, I follow on from my previous blog on AWS CloudWatch Part 1 of 2 to explore how you can go beyond basic agentless AWS CloudWatch monitoring and get detailed insights into EC2 instances by deploying the CloudWatch agent. I&#8217;ll also the key information and planning you need to do this. I\u2019ll [&hellip;]<\/p>\n","protected":false},"author":45,"featured_media":20533,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"_lmt_disableupdate":"no","_lmt_disable":"","footnotes":""},"categories":[391,369],"tags":[518,519,395,566,1095,1096,509,579,403,1434,110,113,115,1440,1091,1092,1090,1088,1089,1093,549,1094,661,633],"class_list":["post-18703","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-aws-monitoring","category-cloud-monitoring","tag-amazon-ec2","tag-amazon-elastic-compute-cloud","tag-aws","tag-aws-cloudwatch","tag-aws-detailed-monitoring","tag-aws-detailed-monitoring-cost","tag-aws-ec2","tag-aws-ec2-monitoring","tag-aws-monitoring","tag-aws-monitoring-tools","tag-cloud","tag-cloud-computing","tag-cloud-monitoring","tag-cloud-monitoring-tools","tag-cloudwatch-agent","tag-cloudwatch-alarm","tag-cloudwatch-cost","tag-cloudwatch-metrics","tag-cloudwatch-monitoring","tag-cloudwatch-pricing","tag-ec2","tag-ec2-detailed-monitoring","tag-ec2-monitoring","tag-elastic-compute-cloud"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Using CloudWatch Agent for Detailed Monitoring of AWS EC2 Instances<\/title>\n<meta name=\"description\" content=\"Learn about the pros &amp; cons of using CloudWatch agent for detailed monitoring of AWS EC2 instances &amp; how eG Enterprise simplifies monitoring.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AWS CloudWatch Monitoring | eG Innovations\" \/>\n<meta property=\"og:description\" content=\"Explore how to go beyond basic agentless CloudWatch monitoring by deploying the CloudWatch agent and learn the key information you need to do this.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/\" \/>\n<meta property=\"og:site_name\" content=\"eG Innovations\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/eGInnovations\" \/>\n<meta property=\"article:published_time\" content=\"2022-03-15T16:23:41+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-04T09:00:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/03\/AWSPart2-Social-Banner.jpg\" \/>\n<meta name=\"author\" content=\"Shashi Shah\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"AWS CloudWatch Monitoring | eG Innovations\" \/>\n<meta name=\"twitter:description\" content=\"Explore how to go beyond basic agentless CloudWatch monitoring by deploying the CloudWatch agent and learn the key information you need to do this.\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/03\/AWSPart2-Social-Banner.jpg\" \/>\n<meta name=\"twitter:creator\" content=\"@eginnovations\" \/>\n<meta name=\"twitter:site\" content=\"@eginnovations\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Shashi Shah\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"13 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Using CloudWatch Agent for Detailed Monitoring of AWS EC2 Instances","description":"Learn about the pros & cons of using CloudWatch agent for detailed monitoring of AWS EC2 instances & how eG Enterprise simplifies monitoring.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/","og_locale":"en_US","og_type":"article","og_title":"AWS CloudWatch Monitoring | eG Innovations","og_description":"Explore how to go beyond basic agentless CloudWatch monitoring by deploying the CloudWatch agent and learn the key information you need to do this.","og_url":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/","og_site_name":"eG Innovations","article_publisher":"https:\/\/www.facebook.com\/eGInnovations","article_published_time":"2022-03-15T16:23:41+00:00","article_modified_time":"2025-07-04T09:00:55+00:00","og_image":[{"url":"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/03\/AWSPart2-Social-Banner.jpg","type":"","width":"","height":""}],"author":"Shashi Shah","twitter_card":"summary_large_image","twitter_title":"AWS CloudWatch Monitoring | eG Innovations","twitter_description":"Explore how to go beyond basic agentless CloudWatch monitoring by deploying the CloudWatch agent and learn the key information you need to do this.","twitter_image":"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2022\/03\/AWSPart2-Social-Banner.jpg","twitter_creator":"@eginnovations","twitter_site":"@eginnovations","twitter_misc":{"Written by":"Shashi Shah","Est. reading time":"13 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/#article","isPartOf":{"@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/"},"author":{"name":"Shashi Shah","@id":"https:\/\/www.eginnovations.com\/blog\/#\/schema\/person\/64a9a35c1bc83011a25804cd80738f29"},"headline":"Using an AWS CloudWatch Agent for System Monitoring of AWS EC2 Cloud Instances","datePublished":"2022-03-15T16:23:41+00:00","dateModified":"2025-07-04T09:00:55+00:00","mainEntityOfPage":{"@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/"},"wordCount":2541,"commentCount":0,"publisher":{"@id":"https:\/\/www.eginnovations.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/#primaryimage"},"thumbnailUrl":"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/AWSPart2-Thumbnail.jpg","keywords":["Amazon EC2","Amazon Elastic Compute Cloud","AWS","AWS CloudWatch","aws detailed monitoring","aws detailed monitoring cost","AWS EC2","AWS EC2 monitoring","AWS Monitoring","AWS monitoring tools","cloud","Cloud Computing","Cloud Monitoring","cloud monitoring tools","Cloudwatch agent","Cloudwatch alarm","Cloudwatch cost","Cloudwatch metrics","Cloudwatch monitoring","Cloudwatch pricing","EC2","ec2 detailed monitoring","EC2 monitoring","Elastic Compute Cloud"],"articleSection":["AWS Monitoring","Cloud Monitoring"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/","url":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/","name":"Using CloudWatch Agent for Detailed Monitoring of AWS EC2 Instances","isPartOf":{"@id":"https:\/\/www.eginnovations.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/#primaryimage"},"image":{"@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/#primaryimage"},"thumbnailUrl":"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/AWSPart2-Thumbnail.jpg","datePublished":"2022-03-15T16:23:41+00:00","dateModified":"2025-07-04T09:00:55+00:00","description":"Learn about the pros & cons of using CloudWatch agent for detailed monitoring of AWS EC2 instances & how eG Enterprise simplifies monitoring.","breadcrumb":{"@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/#primaryimage","url":"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/AWSPart2-Thumbnail.jpg","contentUrl":"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/12\/AWSPart2-Thumbnail.jpg","width":362,"height":235},{"@type":"BreadcrumbList","@id":"https:\/\/www.eginnovations.com\/blog\/aws-cloudwatch-monitoring-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.eginnovations.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Using an AWS CloudWatch Agent for System Monitoring of AWS EC2 Cloud Instances"}]},{"@type":"WebSite","@id":"https:\/\/www.eginnovations.com\/blog\/#website","url":"https:\/\/www.eginnovations.com\/blog\/","name":"eG Innovations","description":"IT Performance Monitoring Insights","publisher":{"@id":"https:\/\/www.eginnovations.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.eginnovations.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.eginnovations.com\/blog\/#organization","name":"eG Innovations","alternateName":"eg innovations","url":"https:\/\/www.eginnovations.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.eginnovations.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2014\/07\/eg-logo-dark-gray1_new.jpg","contentUrl":"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2014\/07\/eg-logo-dark-gray1_new.jpg","width":362,"height":235,"caption":"eG Innovations"},"image":{"@id":"https:\/\/www.eginnovations.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/eGInnovations","https:\/\/x.com\/eginnovations"]},{"@type":"Person","@id":"https:\/\/www.eginnovations.com\/blog\/#\/schema\/person\/64a9a35c1bc83011a25804cd80738f29","name":"Shashi Shah","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.eginnovations.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b37beb2892079dcc6de961de7e947f85?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b37beb2892079dcc6de961de7e947f85?s=96&d=mm&r=g","caption":"Shashi Shah"},"url":"https:\/\/www.eginnovations.com\/blog\/author\/shashi-shah\/"}]}},"modified_by":"eG Innovations","_links":{"self":[{"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/posts\/18703","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/users\/45"}],"replies":[{"embeddable":true,"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/comments?post=18703"}],"version-history":[{"count":0,"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/posts\/18703\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/media\/20533"}],"wp:attachment":[{"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/media?parent=18703"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/categories?post=18703"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.eginnovations.com\/blog\/wp-json\/wp\/v2\/tags?post=18703"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}