How does eG Enterprise Monitor Google Cloud?

eG Enterprise collects metrics from the Google Cloud in an agentless manner only - i.e., using an eG remote agent deployed on any Windows/Linux host/VM in the environment. This agent makes REST API calls to APIs of the services hosted in the Google Cloud project to pull out a wealth of information related to the performance of the services. In order to collect these metrics, the following pre-requisites should be fulfilled:

  • To make the REST calls to the Google Cloud APIs, the eG agent should be configured with name of a JSON key file of a service account in the target project. For this purpose, first, you need a service account with the following roles in the target project:

    • Compute Viewer

    • Monitoring Viewer

    • Cloud Asset Viewer

    If the service account with the mentioned roles already exists in the project, administrators can use the existing service account. If not, administrators should create a service account using the steps explained in the Creating a Service Account section

  • Once the service account is created, download the private key file (in the JSON format) of the service account. The steps for doing this are given in the Downloading Private Key of Service Account section

  • Make sure that the APIs of services (that you want to monitor) in the project are enabled. The APIs are given below:

    • Compute Engine API

    • Cloud Storage

    • Cloud Datastore API

    • Cloud Logging API

    • Cloud Filestore API

    • Cloud Composer API

    • Cloud Billing API

    • Cloud Bigtable Admin API

    • BigQuery API

    • Cloud SQL Admin API

    • Cloud Spanner API

    • Cloud Resource Manager API

    • Cloud Monitoring API

    If not, use the steps given in the Enabling Service APIs section to enable the APIs.

Creating a Service Account

The first step to configure the eG agent to monitor the Google Cloud is to create a service account with the following roles in the target project:

  • Compute Viewer

  • Monitoring Viewer

  • Cloud Asset Viewer roles

To achieve this, follow the steps given below:

  1. Log on to the IAM console by using an Google Cloud account.

  2. If you have multiple projects, select the project where you want to create the service account from the project selector drop down menu (see Figure 1).

    Figure 1 : Selecting a project

  3. Click on the menu icon on the top left corner of the console, then select the Service Accounts option from the IAM & Admin tree in the left-side navigation menu as shown in Figure 2.

    Figure 2 : Selecting the Service Accounts option from the IAM & Admin tree

  4. Selecting the Service Accounts option will invoke the Service Accounts page where you can view the list of service accounts (if any) created for the chosen project. In this page, click on the Create Service Account button as shown in Figure 3.

    Figure 3 : Creating a service account

  5. Enter a name for the service account, and optionally, provide a description in as shown in Figure 4. Next, click the create and continue button to proceed to the next step.

    Figure 4 : Configuring details for the service account

  6. Next, you will be prompted to grant permissions to the service account (see Figure 5) in the Grant this service account access to project section. As mentioned, to monitor the services in the project, the service account should be created with Compute Viewer, Monitoring Viewer, and Cloud Asset Viewer roles. You can do this by selecting a role available for each service API as given in the following table:

    API Role
    Compute Compute Viewer
    Monitoring Monitoring Viewer
    Cloud Asset Cloud Asset Viewer

    Figure 5 : Setting roles for the service account

    Once you set the required roles, press continue button to proceed to next step. If you want the service account to have access to specific resources (e.g., Cloud Storage buckets, Compute Engine instances), you can specify these permissions in the Grant users access to this service account section.

  7. Finally, click Done button to create the service account. The newly created service account will be listed in the Service Accounts page as shown in Figure 6.

    Figure 6 : The new service account

Downloading Private Key of Service Account

To pull useful metrics related to the services, the eG agent needs to be configured with a private key, also known as a service account key that is used to identify and verify a service account. When you create a service account in any of the projects in GCP, you have an option to create a private key for that service account in the form of a JSON file. The private key file includes information such as email address, unique identifier (private key ID), and the private key of the service account. To download the key associated with the created service account, do the following:

  1. Choose the service account for which you want to download the key in the list of service accounts (see Figure 6). Click on the service account name to open its details. This will invoke Figure 7:

    Figure 7 : The details of selected service account

  2. In the Service account details page, navigate to the Keys tab. Under the Keys tab section, you can see a list of existing keys, if any. To create a new key, click on the Add Key drop down and select the Create new key option.

  3. Next, select the Key type as JSON in the prompt that appears and click on the Create button (see Figure 8).

    Figure 8 : Creating the private key

    This will generate the private key file in the JSON format for the service account. The key file will be downloaded to your local system automatically as depicted by the following image.

    Figure 9 : Downloading the keyfile to the local system

  4. Once the key file is downloaded, make sure to store it securely since the private key file contains sensitive information that grants access to your Google Cloud resources.

  5. Then, copy the downloaded key file to the <eG_Install_Dir>/agent/lib folder and provide its name against the private keyfile name parameter while configuring the tests for the target component using the Specific Test Configuration page as shown below.

    Figure 10 : Configuring the tests using the key file

Enabling Service APIs

The eG agent collects the performance metrics of the services by communicating the APIs of the services using REST API calls. Therefore, it is necessary that the APIs for the services should be enabled to collect the metrics. To enable APIs of the services in the chosen Google Cloud project, do the following:

  1. First, select the project to be monitored from the project selector drop down.

  2. Next, expand the API & Services menu option from the left-side navigation pane. Select Library from the expanded menu as shown below.

    Figure 11 : Selecting the Library

  3. This will lead you the API Library page where you can view a list of APIs of available Google Cloud services. To find out the API of a service that you wish to monitor, you can specify whole or part of the service's name and as shown in Figure 12.

    Figure 12 : Searching the API of service that is to be monitored

  4. Then, click on the API in Figure 12 to open its Product details page.

    Figure 13 : Enabling the Google Cloud service

    On the Product details page, click on the Enable button to enable the service. Note that the eG agent will be unable to report metrics if the service is not API-enabled in the chosen project.

Assign roles to other projects

eG Enterprise allows administrators to monitor other projects using the service account that they created in the Google Cloud project to be monitored. To enable monitoring of other projects, you need to grant access to the service account of the target project to access the services in other projects and assign roles to the projects. To achieve this, follow the steps given below:

  1. Login to the Google Cloud Console using one of the following roles:

    • Project IAM Admin (or higher)

    • Folder Admin

    • Organization Admin

  2. In the Google Cloud console, go to the IAM page.

  3. Select a project from the project selector drop down menu.

  4. To grant a role to a principal that does not already have other roles on the resource, click Grant Access (see ), then enter the email of the service account you created earlier.

    Granting Access to Service Account

    Figure 14 : Granting access to the service account

    Then, assign the following roles as shown in Figure 15:

    • Compute Viewer

    • Monitoring Viewer

    • Cloud Asset Viewer

    Assigning Roles to Projects

    Figure 15 : Assigning roles to the project

  5. Finally, click Save to register the changes.