Logging and Auditing in IT Monitoring Tools

It is critical that access to any configuration changes or management actions made to monitoring platforms are logged and traceably audited. In this article, I will help you learn how to discover the auditing capabilities in IT monitoring tools. You will learn how to audit and manage the monitoring platform itself and make sure that it is being used appropriately. In a future article, I intend to focus more on how to leverage and evaluate features in monitoring tools to audit and track configuration changes of the applications and systems they monitor – e.g., auditing the use of scripts and remote control actions from the tool and automatically tracking changes such as adding a new hypervisor or server to the infrastructure landscape.

As an enterprise monitoring solution, eG Enterprise offers a powerful platform to provide unified end-to-end monitoring of applications, infrastructure, and third-party services by leveraging APIs (Application Programming Interfaces) and supported interfaces. As with any monitoring tool, care must also be taken to control permissions and privileges and access to the tool itself to ensure that the deep insights provided are only accessed by the authorized staff in a traceable and auditable manner. Role Based Access Control (RBAC) features are a de facto standard for monitoring tools.

Moreover, it is best practice to ensure proactive auditing detects and eliminates any rogue or malicious operators operating from outside or even from within the organization. Many freeware and entry-level tools are woefully inadequate in this area, failing to even record logins to the monitoring platform let alone configuration changes made to how monitoring is performed or what is to be monitored. Depending on which industry you are in, you may have to comply with industry standards, such as HIPAA, SOX, etc., that may require you to have controls over all the tools you have in place. Auditing is an important way in which you prove compliance with these standards as far as a monitoring tool is concerned.

The consoles, dashboards and reports associated with a monitoring platform are relied upon by organizations within critical business processes and relied upon by individual employees to perform specific tasks and roles. Uncontrolled or untraceable changes to your monitoring configuration and interfaces can impact on your staff’s work routines and, ultimately, the ability of a business to deliver highly available applications and services to end customers and employees. While auditing is mainly focused on admin changes, you can also use it to track new dashboards or dashboard templates being created or being manipulated.

Auditing is not just for security. It also helps determine why the monitoring tool is sometimes misbehaving or baselines and frequency of alerts have changed – e.g., if someone changes thresholds incorrectly. Auditing changes can alert you to changes that should not have been made or users who may need additional training and allows the ownership of decisions to be tracked and decisions questioned.

When evaluating a monitoring platform, you should consider what audit logs are supported, the coverage of the auditing, and how easily you can access and automatically audit the data collected without resorting to manual inspection and parsing of text files. The problem is more severe in a SaaS configuration, where the management console is accessible over the Internet, making it more vulnerable to external attackers.

Top Auditing Features for Monitoring Platforms and Tools

1. Auditing of Logins to the Monitoring Platform

A significant failing in many freeware or entry level products is the failure to even log or audit user access to the monitoring product, and for any enterprise-grade organization, this is usually a red flag to adoption because of the compliance implications. eG Enterprise records each and every administrator login and provides out-of-the-box audit reports that allow both proactive monitoring and forensic analysis of user behaviors and actions performed. Failed logins are also audited assisting the detection of many types of malicious attempts to breach security measures.

At a minimum for a successful login, data to be captured includes:

  • the name of the user
  • the IP address of the host from which the user accessed the eG management console
  • the exact time of login
  • the accurate time of logout
  • the duration of the user access

For failed login attempts, at a minimum, data capture should include:

  • the name of the user
  • the IP address of the host from which the user attempted to login to the eG management console
  • the Interface type that was used – whether web or command line
  • the exact time of the login attempt
  • the reason for the login failure
Critical Feature: MUST monitor and audit each and every login to the monitoring platform and MUST monitor failed login attempts.
Granular controls should allow control of who can access all audit log data. It must be possible to restrict users to see no audit data or only their own and allow only designated authorized admins access to overviews of all users’ data.

For further details on eG Enterprise auditing of successful logins, please see: Auditing Successful User Logons (eginnovations.com)

For further details on eG Enterprise audit reporting on failed logins, please see: Auditing Failed Logons (eginnovations.com)

From the eG Enterprise console, select the “Admin” tab and from the menu select “Audit” to show the range of Audit reports available.

Auditing IT Monitoring tools

Figure 1: If you have the appropriate administrator privileges, auditing will be accessible via the “Admin” tab of the main eG Enterprise console.

If you select the “Successful Login” report, you will be given an interface to configure your audits to your own needs.

Logon report from the IT monitoring tool

Figure 2: “Successful Login” report can be configured to your own needs.

All the columns are active to allow you to sort and order the data by key parameters, such as date or duration of logon.

Auditing user logons in IT monitoring tool

Figure 3: A range of parameters are available to control the reports generated.

The timeline of the report can be controlled to the granularity of your choice, from hours to days, weeks, months, or longer; the time range of interest is also fully adjustable. And filters can be applied to limit the view to an individual or group of users. The “Interface” used to access eG Enterprise is another filter allowing you to separate the auditing of user access from web console or via any API/command line interfaces.

Selecting the “Failed Logins” Report from the Admin->Audit menu will access similar data for failed logins including reasons for failure.

Failed logons report is very useful for identifying security issues

Figure 4: Troubleshooting failed logins becomes simple (it is often a mistyped or forgotten password, deleted account, or misremembered username), but failed logins can be a sign of malicious brute force attacks especially if multiple login attempts occur from different IPs or strange IPs. Details on locking down access to eG Enterprise via IP address can be found in Administration Policy (eginnovations.com).

2. Auditing Configuration Changes made to the Monitoring Tool

eG Enterprise provides AUDITLOG REPORTS, in which you can keep tabs on critical configuration changes made using the eG administrator interfaces, such as password changes, test parameter changes, new server additions, threshold changes, etc., which can significantly alter the way the eG Enterprise system performs monitoring. Sometimes, these configuration changes, if not done properly or if carried out by unauthorized/unqualified personnel, can cause the eG Enterprise system to generate false alerts and perform inaccurate diagnosis. As these AUDITLOG REPORTS reveal what admin settings were modified by which user, along with the details of the original settings, they greatly help administrators in quickly identifying and rectifying errors (if any) in configuration.

Details of reports provided by eG Enterprise can be found in the documentation (see Auditing Configuration Changes made using the eG Administrative Interface) and include:

  • the date/time of the change
  • the name of the user who made the change
  • the IP address of the host from which the user accessed the eG admin interface
  • the module that was accessed by the user
  • the specific operation/activity that was performed by the user on that module
  • the Interface type that was used – whether web or command line
  • the detailed description of the change, followed by a snapshot of the settings prior to change, and the settings after the change; if a configuration has been newly introduced (e.g., a server has been newly managed), then only the Current Settings will be displayed. By default, every change record that the report displays will be accompanied by the Current and Previous configuration settings, this can be modified to simplify reports if desired.
Note: eG Enterprise can be deployed with clusters of managers to provide built in redundancy for failover. All auditing features are designed to work and trace operations in the normal way in such a configuration. Many freeware and entry-level products do not provide full auditing in redundant configurations, and this is an item to be added to any evaluation checklist as to whether actions can truly be associated and traced with multiple managers and management clusters.

For full details, see Auditing Configuration Changes made using the eG Administrative Interface.

Expert tip: always check that full auditing is implemented for monitoring product even in failover architectures, such as multiple managers and management clusters.

The “Audit->Admin” menu item gives access to changes made; these include critical data on the creation of new users, changes to privileges, etc. Auditing logins alone is insufficient if this critical data is not traceable to ensure those authorized are done so by legitimate operators and the possibility of rogue ex-employees granting and leaving backdoor accounts can be eliminated.

Audit log of all changes made to an IT monitoring tool

Figure 5: Customizing the admin report view. Again, timelines, date ranges, user, and interface filters are available. Additional filters can be applied for Host IP, Modules, and specific Activities. By default, only those modules and activities applied within the reporting time period are shown to accelerate deep dive analysis and eliminate null and empty reports when filtering is applied.

Filtering on the “Add User” activity, you can review whether new users have been configured correctly and, if not, by whom. Any changes to permissions, as well as who made them, when, and from where, will be recorded on-going.

Audit report of all changes

Figure 6: Reviewing the creation of a new account made for me by the user “admin”, it appears the admin has restricted my ability to see reports in Reporter to a 2-week time period for increased data security. Additionally, I am not trusted to delete alarms in the system, as that is the helpdesk operators’ job, and they certainly don’t want me interfering! The admin has also defined a “MMM dd, yyyy” format to be used, such as “May 10, 2022”, as this system is used across multiple geographies, and we like to avoid USA/Europe date format confusions 😀.

All changes that modify what is monitored, how it is monitored, and the users view from eG Enterprise of the monitoring are recorded.

Audit manual as well as automated changes

Figure 7: All components added to the monitoring infrastructure are logged, whether added manually or via intelligent auto-discovery technologies.

Auditing threshold changes made in the IT monitoring tool - through UI and via API or CLI

Figure 8: Threshold changes are captured. Here, the audit logs record that a user has modified the sensitivity of a “Disk busy” metric threshold and when and how often alerts will be modified. The time window in which a threshold breach is considered significant has also been changed. Such changes will affect reports on the number of alarms so they should always be tracked and transparent to management.

Similar auditing and reporting functionality is provided for all eG Enterprise user roles and modules, see:

  • Auditing Configuration Changes made using the eG Monitor Interface (Just like changes made using the eG admin interface, care should also be taken while making changes using the eG monitor interface – e.g., while deleting alarms, acknowledging alarms, configuring quick insight/live graph views, etc. Changes that are implemented carelessly can only add to an administrator’s confusion and cause unnecessary delay in problem resolution.)
    Figure 9: Changes to the monitor interface include modifications to dashboards. Sometimes the history and design rationale behind a dashboard or custom widget may be lost and rigorous auditing assures that ownership can be traced.
  • Auditing Configuration Changes made using the eG Reporter Interface (Typically, the key configuration changes that a user can make using the eG Reporter component is to add/modify/remove FAVORITES and SCHEDULE report configurations.)
  • Auditing the Display Settings Changed Using the eG Configuration Management Interface (Generate audit log reports that will help you instantly identify whether any changes were made to the dashboard and overall display settings of the eG Configuration Management interface, as well as who made these changes, and when.)

3. Auditing of Automated and Scripted Actions and Bulk Operations

Always check auditing covers API (Application Programming Interface), Bulk operations, Auto-discovery by agents, and Command Line access to your monitoring tools beyond web and console access.

Auditing helps track automatic actions taken by the monitoring tool – e.g., deleting expired users and all their components; these types of bulk and self-service actions and maintenance tasks are heavily utilized by our MSP (Managed Service Provider) customers.

Automated programmatic and scripted access is often an organization’s Achilles’ heel for secure monitoring platform access!

4. The Ability to Export Audit Reports in Flexible Formats and Automate the Scheduling Audit Reporting for Compliance

For systematic by-process due diligence, configuration change tracking and auditing must remove the need to manually trawl audit logs; proactive monitoring should be in place and regular automated audit report generation and archiving implemented. Log data should be available via a flexible GUI, including search and filtering, and ideally via .CSV and .pdf formats for additional analysis. It must be possible to capture and retain data on significant timescales to enable retrospective compliance reviews and future forensic diagnosis.

All audit reports are presented in .html format with an option to export in .pdf or .csv formats, to print or to schedule and email.

5. Full Auditing Available Regardless of the Deployment Architecture Chosen for eG Enterprise – On-premises, in Cloud, and SaaS Ready-to-go

eG Enterprise can be deployed on-premises, in a cloud of your choice, or used via our ready-to-go SaaS offering; details of the diverse options, including options for MSPs (Managed Service Providers), are covered in: Deploying eG Enterprise Monitoring – SaaS or On-Premises. Unlike many competitors, our comprehensive auditing is available for all options, including on-premises, as well as SaaS because we understand that those choosing on-premises for specific compliance often have stringent auditing requirements too.

Ensuring Security within Monitoring Architecture

Comprehensive auditing capabilities in IT monitoring is essential. Tools should offer logging and log management is just one of many security and governance features we include within eG Enterprise that are widely used in corporate enterprises and organizations that are subject to regulatory control, including limiting access to approved locked-down IP addresses and ranges, multi-factor authentication, single sign-on, account locking, encryption, a no agent listening on open ports architecture, and more. (See Security Architecture of eG Enterprise.)

Further Information