In an article a few months ago, my colleague covered the functionality within eG Enterprise that ensures secure and traceable audit trails for both users and admins of eG Enterprise allowing automated auditing and reporting for regulatory compliance and security, see Auditing Capabilities in IT Monitoring Tools | eG Innovations. Today, I will follow from this article and cover how eG Enterprise also controls and audits the execution of Remote Control Actions and scripts. These features ensure that privileged access and actions are traced and auditable with the added benefit that changes and actions that have unexpected negative side effects can be identified and rectified.

What are Remote Control Actions?

When eG Enterprise detects any issues in the target application or infrastructure, an administrator may want to immediately rectify them. For example, the admin may want to kill a runaway process that is taking up a lot of CPU. This is where Remote Control Actions come in.

Administrators can use remote control actions to investigate issues in greater detail, to rectify issues and indeed automate certain remediation steps. Common actions include:

  • Rebooting or powering off servers
  • Shadowing user sessions and taking screenshots of their sessions for support reasons
  • Disconnecting idle sessions
  • Collecting data from the session remotely for diagnostic purposes
  • Executing scripts or commands remotely for diagnostic or remediation e.g., “Kill user GPO (Group Policy Objects) policies”

How do Remote Control Actions work in eG Enterprise?

One of the challenges in eG Enterprise is its secure architecture. eG agents do not listen on any TCP ports, so when a user wants to execute activities on a system, the eG manager does not have a way to directly connect to an agent and control it.

Remote control is implemented using an intelligent polling mechanism. When an agent communicates with the manager to report performance data, the eG manager piggybacks the response with instructions to start remote control. Once an agent is in control mode, it polls the manager frequently, so that commands provided by admins are instantly executed on the agent systems. This mechanism ensures that no additional TCP ports are used for remote control and admins still have a mechanism to execute remote commands on the target systems.

When it gets instructions from the eG Enterprise interface from an admin, through the eG manager, an agent executes the instructions, as long as it has permissions to do so and the result is communicated back to the eG Enterprise web console.

Why Audit and Control Remote Control Actions?

Many of these actions are powerful and invasive and if used inappropriately can affect the services being delivered (e.g., rebooting servers), or they can compromise data security or user privacy (e.g., taking a screenshot of a user session). Beyond strict Role Based Access Control on privileges to use actions, organizations also need robust tracing and auditing capabilities to ensure the actions of a rogue or malicious employee (Insider Threats) or a rogue script are deterred and detected.

Specific scenarios organizations need to consider include:

  • Many organizations do not want L1 level support staff, especially if employed on a casual or contracted basis, to be able to access and see user desktops or perform administrative tasks such as reboots.
  • MSPs (Managed Service Providers) offering multi-tenancy and tenant self-service will usually need fine grain control of RCAs and the ability to demonstrate secure access and auditing of actions to meet customers compliance requirements.
  • Quick fixes such as rebooting a server regularly can resolve numerous issues but mask a serious underlying root cause. The ability to audit and review such actions is important.
  • Operations such as session shadowing and screenshotting a user’s screen need to be tightly controlled and when undertaken properly tracked and auditable on-going. This is especially important in geographies such as Germany where state employee privacy protections beyond GDPR may exist.
  • For many compliance standards and regulation audit data must be available long beyond the default retention timelines of most cloud platforms and services.
  • Often security breaches or the actions of malicious ex-employees only become apparent several months or years after the event. The ability to prove criminal or illegal behavior in forensic investigations after the event is important.
  • If multiple staff are involved in support calls, the ability to verify what has been undertaken and by whom and when can avoid miscommunications and errors and remove a blame culture.

Controlling Privileged Access to Remote Control Actions (RCA) in eG Enterprise

When adding/modifying a user in eG Enterprise, an admin can enable/disable control actions for a user – i.e., the user could be either associated with all control actions or none. Many organizations usually choose to disable all control for frontline L1 operators.

Beyond this, newer versions of eG Enterprise allow customized access to every single RCA on a per user basis. This could mean a L3 help desk operator could be allowed to session shadow and screenshot a user’s session for diagnostic purposes but would not have privileges to run scripts on the user desktop or perform heavyweight system administration tasks such as server reboots.

Figure 1: eG Enterprise provides a simple GUI too customize the commands a user can action – selecting the radio button “Customize actions”

Having selected “Customize Actions” a simple GUI allows fine-grained control over what actions are allowed for each user.

Figure 2: Configuration on a per action, per user basis ensures granular and controlled access for security whilst empowering teams with the tools they need to perform their individual role

How to Access Remote Control Audit Data

Audit data for RCA and similar data is restricted to administrator roles and accessible via the eG Enterprise “Admin” tab. RCAs (Remote Control Actions) and other operations performed by those using the “Monitor” tab are found under Audits -> Monitor. In an equivalent way Audits -> Admin will access audits on administrator operations.

Figure 3: Where to find the Audit trail for RCAs actioned by eG Enterprise users

Selecting “Audit -> Monitor” will present the user with a filter menu. This smart filter menu will only present what data is available. Select an appropriate timeline on the filter

Figure 4: Note how on this system over the last month only the modules where changes have been made and the activities recorded are presented by smart-filtering, ensuring the user does not run blank or null queries. In this case I limited the filter to only view Remote Control Actions.

Filters can be applied to investigate specific users or interfaces (web console, command line, bulk operations, agent discovery operations and so on). It is important that any bulk or automated operations triggered by users are captured within audits.

Running the audit, I can see the records arising from support call interactions.

Figure 5: Here the administrator of an account “rdsdemo” responsible for a Citrix demo farm has been investigating a user logon issue. During the support interaction they investigated the GPOs (Group Policy Objects) in use and performed invasive activities such as taking screenshots.

Ensuring activities such as screenshotting and session shadowing are only performed by authorized staff and furthermore only when appropriate by those staff is essential within an accountable organization.

Other records show operations of the super-privileged “admin” account viewing data on systems and running commands.

Figure 6: Highly privileged Admin activities are all recorded for later scrutiny and audit.

Beyond security and compliance audit, most administrators find this functionality extremely useful for improving their working practices and inter-employee communication. Often during a support incident, it is extremely useful to review in what order and when actions were taken, especially if a second opinion is sought or an incident is handed over to a different team or an employee takes over from the previous help desk shift.

Custom Remote Control Actions

It is also possible to add custom RCAs associated with your own bespoke scripts or commands. By default, recent versions of eG Enterprise will require scripts to be signed for additional security. Information on how to do this is covered in the documentation, see: Control Actions (eginnovations.com).

eG Enterprise is an Observability solution for Modern IT. Monitor digital workspaces,
web applications, SaaS services, cloud and containers from a single pane of glass.

Further Information