Storm Topologies Test

Spouts and bolts are connected together and they form a topology. Real-time application logic is specified inside Storm topology. In simple words, a topology is a directed graph where vertices are computation and edges are stream of data.

A simple topology starts with spouts. Spout emits the data to one or more bolts. Bolt represents a node in the topology having the smallest processing logic and the output of a bolt can be emitted into another bolt as input. Storm keeps the topology always running, until you kill the topology. Apache Storm’s main job is to run the topology and will run any number of topology at a given time.

This test reports the number of workers, executors and the tasks completed in the topology of the target Apache Storm. Using this test, administrators would be able to identify the status on whether/not the topology is active. Any abnormal condition of the topology will alert the administrators to take preventive action against the abnormal condition.

This test included in the eG agent also monitors the uptime of topology before users start complaining.

Target of the test : Apache Storm

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for each Topology in the target Apache Storm.

Configurable parameters for the test
Parameter Description

Test period

How often should the test be executed.

Host

The IP address of the target server that is being monitored.

Port

The port number through which the Apache Storm communicates. The default port is 8080.

SSL

By default, the SSL flag is set to No, indicating that the target Apache Storm is not SSL-enabled by default. To enable the test to connect to an SSL-enabled Apache Storm, set the SSL flag to Yes.

Measurements made by the test
Measurement Description Measurement Unit Interpretation

Status

Indicates the status of the topology in the target Apache Storm.

 

The values reported by this measure and its numeric equivalents are mentioned in the table below:

Measure Value

Numeric Value
Killed 70
Rebalancing 80
Inactive 90
Active 100

Note:

By default, this measure reports the Measure Values listed in the table above to indicate the status of the topology in the target Apache Storm.

Total workers

Indicates the total number of workers in Storm topology.

Number

 

Total executors

Indicates the total number of executors in Storm topology.

Number

 

Total tasks

Indicates the total number of tasks in Storm topology.

Number

A very high value is required for this measure.

Replication count

Indicates the number of nimbus hosts on which this topology code is replicated.

Number

 

Requested total memory

Indicates the size of requested memory for Storm topology.

MB

 

Assigned total memory

Indicates the size of assigned total memory for Storm topology.

MB

 

Requested memory on heap

Indicates the size of requested heap memory for Storm topology.

MB

 

Assigned memory on heap

Indicates the size of assigned heap memory for Storm topology.

MB

As a rule of thumb, there should be enough heap memory to address data processing. If the heap memory is high, then it leads to garbage collection.

Requested memory off heap

Indicates the size of requested memory off-heap for Storm topology.

MB

 

Assigned memory off heap

Indicates the size of assigned memory off-heap for Storm topology.

MB

If there is no off -heap memory, then you will be unable to perform the data processing. To avoid this, it would be good practice to allocate adequate off-heap memory.

Requested CPU core

Indicates the number of requested CPU core for Storm topology.

Number

 

Assigned CPU core

Indicates the number of assigned CPU core for Storm topology.

Number

 

Uptime

Indicates the time period that the topology has been up since the last time this test ran.

Hrs/Mins/Secs

The accuracy of this metric is dependent on the measurement period - the smaller the measurement period, greater the accuracy.