How does eG Enterprise Monitor Apache Storm?

eG Enterprise employs agent-based approach to monitor the Apache Flume. For agent-based monitoring of Apache Flume, an internal eG agent should be deployed on the server hosting Apache Flume.

eG Enterprise uses Java Management Extensions (JMX) to connect to target Apache Flume server and collect relevant metrics.

Pre-requisites for Monitoring Apache Flume using eG Enterprise

  1. After extracting the Apache Flume package downloaded from "https://flume.apache.org/download.html". You will find the "flume-env.ps1" in "D:\apache-flume-<version>\apache-flume-<version>-bin\conf". By default, the JMX automatic configuration is disabled. To enable it, configure the following environment variables:

  • For Windows select flume-env.ps1, and configure the JVM_OPTS variable. For example:

    JVM_OPTS="-Xms100m -Xmx200m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.rmi.port=6000 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=6000 -Djava.rmi.server.hostname=172.16.14.98"

  • For Linux select flume-env.sh, and configure the JVM_OPTS variable. For example:

    JVM_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=6001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=172.16.14.98"

  1. To find the JMX port, open flume-env.ps1, you can check or update the JMX port and other details in JVM_OPTS parameter.

Figure 1 : Apache Flume JMX port and other configurations