Full-Stack Kubernetes Monitoring with eG Enterprise

Full-stack Kubernetes monitoring with eG Enterprise provides end-to-end visibility into your containerized environments. It can auto-discover and auto-manage Kubernetes components including clusters, nodes, and applications. eG Enterprise addresses the need for administrators who are looking to fully monitor Kubernetes environment including:

  • Monitor the Kubernetes Clusters.

  • Monitor different types of nodes in the cluster viz. master nodes, worker nodes, infra nodes.

  • Monitoring the applications running in the containers, databases, Java applications etc.

The monitoring model also addresses the needs of different set of stakeholders by providing a way to auto discover and auto manage the component with just the basic configuration. The stakeholders include -

  • The Operations Team: The team includes the support and administrators who are responsible to ensure that the cluster is working ad the nodes are in working condition, looking for early warning indicators to preempt ay failure.

  • The DevOps team: The team is responsible for defining the cluster architecture and handle the creation of namespaces, PODs, etc.

  • The Application Teams: The teams which are responsible for developing, deploying and maintaining the applications running inside the containers.

eG Enterprise aims to provide a full stack view of entire Kubernetes environment, providing complete visibility into the health of nodes, cluster and apps, ad addressing the need of all stakeholders.

The eG Enterprise monitoring model provides a set of comprehensive capabilities to monitor entire Kubernetes stack and not just the Kubernetes worker. The administrators don't have to setup separate monitoring for Cluster, and nodes, eG Enterprise provides an automated approach to discover the Kubernetes components in the network and allow for the automated configuration of monitoring all at once for Kubernetes Cluster, Master and Worker nodes, and Kubernetes application hosted in the container. The cluster is monitored using agentless approach while the agent is required for monitoring the master and worker nodes. This model massively simplifies the monitoring setup process of monitoring Kubernetes and Kubernetes platforms like Openshift, Rancher, VMware VSphere etc, as well as cloud based Kubernetes platform like Amazon Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), IBM Kubernetes Service (IKS), Google Kubernetes Engine (GKE) etc. All of these components follow the same monitoring approach and eG Enterprise monitoring model.

eG Enterprise Kubernetes monitoring model provides an integrated approach for full stack monitoring of entire Kubernetes environment, covering clusters, nodes, and containerized applications. It also offers selective monitoring which allows users to focus on monitoring specific Kubernetes components such as clusters, nodes, or individual namespaces separately. Users can choose either approach based on their preference and monitoring focus. The Kubernetes cluster and its associated nodes will each consume one Premium License, while container applications will consume a separate Container App License. For selective monitoring refer to Selective Kubernetes Monitoring

Now, to start monitoring, a set of prerequisites needs to be fulfilled, and the steps to set up auto-discovery and manage the monitored components are provided in this section.

Prerequisites for monitoring Kubernetes Cluster

Follow the steps below to ensure environment is set up for monitoring. These steps are only applicable for full stack monitoring.

  1. eG Container Agent Network Connectivity: The eG Container Agent running on all master and worker nodes must have network access to the eG Manager.

  2. Access for eG Agent Operator and Custom Resource Installation: When installing the eG Agent Operator and eG Agent Custom Resources, the following command retrieves YAML definitions directly from the eG Manager over HTTP/HTTPS.

    oc apply -f "<eG-Manager-URL>/final/egodac?..."

    Therefore, the kubectl/oc client or the machine executing these commands must have network access to the eG Manager URL.

    If direct access is not possible:

    1. Use the “Download YAML” option in the eG console.

    2. Copy the downloaded YAML files to a location accessible by your kubectl/oc client.

    3. Apply them manually using kubectl apply -f <file> or oc apply -f <file>.

  3. kubectl/oc Client Access to the Cluster: To deploy and manage eG Agent components, the user must have kubectl (for Kubernetes) or oc (for OpenShift) access to the cluster.

    Requirements for the executing user:

    1. A properly configured kubeconfig or an active oc login session pointing to the target cluster.

    2. Cluster Administrator privileges (or equivalent) to create Namespaces, Deployments, DaemonSets, CRDs, ClusterRoles, and ClusterRoleBindings.

    3. The kubectl or oc CLI tools must be installed and accessible from the environment executing the commands.

  4. Container Agent Image Access: By default, container agent images are pulled from public registries such as Docker Hub. If the environment does not allow public internet access, images must be copied to a private registry and referenced from there.

  5. Cert-Manager Installation: Cert-manager is a Kubernetes add-on that automates the management and issuance of TLS/SSL certificates in Kubernetes cluster. Cert-manager must be installed and available in the cluster. For this purpose, cert-manager addon is required to be installed inside of Kubernetes cluster, which generates and provide certs.

    Sample Installation guide: https://cert-manager.io/docs/installation/kubectl/#1-install-from-the-cert-manager-release-manifest

  6. Metrics Server Installation: Metrics Server is a cluster-wide aggregator of resource usage data, such as CPU and memory. It collects metrics from the Kubelets running on each node and exposes them through the Kubernetes API. To know how to install a Metrics Server in the cluster, refer Installing the Metrics Server.

  7. Licensing Requirements: Adequate licenses (Premium and Container Apps) must be available. Without sufficient licenses, one or more components may not be auto-discovered or auto-managed.

Before setting up the Kubernetes Cluster for monitoring, make sure that discovery is enabled for Kubernetes cluster.

  • The new Kubernetes monitoring model takes enables the eG Enterprise to discover the components automatically without the need to manually discover and add components. eG Enterprise is capable of searching your network and finding interconnected stack of Kubernetes components. Once you have discovered various components like Cluster, Master and Worker nodes, applications deployed on Kubernetes containers, you can choose to monitor all of them or selectively monitor specific component. From the eG manager console, go to Admin->Discovery->General, make sure that all the Kubernetes components,(Cluster, Nodes and Container applications) are enabled for discovery. These components will be enabled by default. To know more about the discovery process, refer Discovering Containerized Applications Using the eG Agents.

  • Then, go to the Auto manage section, confirm that the clusters, nodes, and container applications are included to be auto managed after discovery. These components will be enabled by default. To know detailed steps, refer Auto-management of Discovered Containerized Applications

  • Now, navigate to Auto delete section and verify the auto delete durations for the Kubernetes entities. By default, the auto delete durations are set to 8 days for clusters, 4 hours for nodes and 15 minutes for container applications. If needed, you can edit these durations based on your environment. For detailed discussion, referAuto-unmanaging or Deleting Kubernetes Nodes and Auto-unmanaging or Deleting Containerized Applications

Setting up auto discovery for Kubernetes Cluster

The first step towards managing the Kubernetes environment using eG Enterprise is start the discovery process. Before you even start the discovery process you need to decide whether you want to monitor the entire Kubernetes stack or a particular component. In most of the cases, setting up the Kubernetes stack holistically will make more sense, in all of those cases select full stack monitoring. However, in certain cases when you don't want to monitor entire stack and want to go with specific component, you can go ahead with selective monitoring. The Discover/Monitor interface in eG Enterprise provides these options and high degree of flexibility in configuring the monitoring on the Cluster, nodes or application. It provides multiple ways in which you can setup your nodes for eG agent installation and you can choose whichever you are comfortable with, whether using Helm or Operator Manifest eG enterprise automatically generates the script or deployment artifacts for you to run on your nodes.

If you want to monitor entire Kubernetes stack then select Full Stack Monitoring, below steps describe how to set up Full Stack Monitoring.

Follow the steps below to set the full stack monitoring:

  1. Go to Admin Dashboard -> Discover/Monitor.

  2. From the page titled What would you like to Discover/Monitor?, Select Kubernetes and Containers.

  3. From next page select Kubernetes, Rancher, Openshift, EKS, AKS, Tanzu, Google, you will land on below page as shown in Figure 1

    :

    Figure 1 : Monitoring Kubernetes Cluster

  4. Choose FULL STACK MONITORING option - Kubernetes Cluster, Nodes, Applications. Next you'll get the options to specify the cluster and cluster type as shown in Figure 2.

    Figure 2 : Adding Kubernetes Cluster

     

    • In What do you want to monitor section, select all three option if you want to set up full stack monitoring as shown in Figure 2

    • If you want to monitor only nodes, select Container Engine - Master/Worker Node.

    • Select Containerized Application if you want to monitor apps. Also, select Auto Instrument APM if you want to monitor business transactions of the containerized applications being monitored. This is shown in Figure 2

    • Click Next.

  5. In Kubernetes preferences for Kubernetes cluster, provide the cluster name, and select Worker and Master if you want to monitor worker and master nodes. This is shown in Figure 3.

    Figure 3 : Specifying the cluster details

  6. In Application Monitoring and Instrumentation section, provide the details as shown inFigure 4 and described below.

    Figure 4 : Monitoring and Instrumentation details

    • Namespaces to be considered: Specify the name of the one or namespaces separated by comma. You can also provide wildcard patterns.

    • Namespaces to be ignored: Specify the names of namespaces to be ignored separated by commas. You can also provide wildcard patterns.

    • Manage process patterns: Click the hand icon to check the process patterns, and ensure the pattern for process you want to monitor is there in the list. If not you can add/update the pattern.

      Figure 5 : Process patterns for container application monitoring

    • Auto Instrumentation rules: Configure the rules by clicking edit icon.

      • On Auto Instrument Rules prompt, click on Add Rules. Specify the Rule Name, Pod label details of the pod where you want the instrumentation to run, and select container from Container Selection list. This is shown in Figure 6

        Figure 6 : Auto instrumentation rules list

      • Click Add to add the rule to the list as shown in Figure 7.

        Figure 7 : Adding auto instrumentation rules

      • Click on Update to update the list of instrumentation rules.

  7. In Container Image details section, provide the details as shown in the Figure 8, and described below.

    Figure 8 : Specifying the details for images

    • Registry URL: Provide the URL of image registry from where the images will be pulled. It can be public or private registry,

    • eG Universal agent operator image: Provide the name and version of eg agent image.

    • Helm Chart Version: Helm chart is a package manager which bundles all Kubernetes resources, Specify the version of Helm chart being used.

    • Container agent image: Specify the name of image for container agent.

    • Java profiler image: Java profiler will be used for Java based application instrumentation. Specify the name of Java profiler.

    • NodeJS profiler image: NodeJS profiler will be used for NodeJS based application instrumentation. Specify the name of NodeJS profiler.

    • Agent JVM memory: Specify the amount of memory which will be available for use for Agent JVM.

    • Agent memory request: Specify the amount of memory agent can use for storing requests,

    • Agent memory limit: Specify the limit for memory which agent can use from pod.

    • Agent CPU request: Specify how much CPU time, the agent can consume for request processing.

    • Agent CPU limit: Specify the limit of CPU usage for agent.

    • Types of workload to auto instrument: Specify what kind of workloads you want to monitor through auto instrumentation, select Replicaset, or Statefulset or both.

  8. Click Next, from the next screen copy the auto generated command text as shown in and run to install eG agent. You can choose either of two approaches, Helm or operator manifest.

Figure 9 : eG agent installation