Top Conversations Test

Bandwidth-intensive conversations between two hosts over the network is one of the reasons why users may experience slowness when communicating over the network. To resolve this, network administrators must first keep a close watch on the NetFlow between each pair of hosts that is interacting over the network, study the bandwidth usage of and data exchange between each pair, and identify the pair that is engaged in an bandwidth-intensive conversation. This is what the Top Conversations test helps administrators achieve!

This test auto-discovers the top-n conversations in terms of the amount of data exchanged over the network. For each conversation, this test reports the amount of data exchanged between the hosts engaged in that conversation and the bandwidth consumed in the process. This way, the test points administrators to the pair of hosts that are engaged in bandwidth-intensive communication.

Target of the test : A Netflow Device

Agent deploying the test : An external agent

Outputs of the test : One set of results for each of the top-n conversations (in terms of traffic volume) happening over every network interface supported by the target device

First-level descriptor: Interface name

Second-level descriptor: IP addresses of the hosts engaged in an conversation over the network. Format is <SourceHost> to <DestinationHost>

Metrics are also reported for an Others descriptor. Netflow statistics pertaining to all conversations that are not the top-n conversations are aggregated and presented against the Others descriptor for each interface.

Configurable parameters for the test
Parameter Description

Test period

How often should the test be executed

Host

The host for which the test is to be configured.

Filter By, Minimum Flow Percent, Report Top N Flows, Show DD for Top N Flows

By default, for each interface that the target Netflow device supports, this test reports metrics for only the top-10 conversations that handle over 3% of the total traffic on that interface. Also, by default, the test will report detailed diagnostics for only the top-5 conversations that handle over 3% of the total traffic on that interface. This default setting enables network administrators to focus on the abnormal flows alone. This behavior is governed by the default setting of the Filter By, Minimum Flow Percent, Report Top N Flows, and Show DD for Top N Flows parameters. By default, the Filter By option is set to %Traffic and the Minimum Flow Percent is set to 3 (%). You can increase or decrease the Minimum Flow Percent depending upon the level of traffic that you deem as abnormal. Also, by default, the Report Top N Flows is set to 10 and Show DD for Top N Flows is set to 5. You can increase or decrease the value of these two parameters depending upon the level of visibility you require.

You can also override the Filter By default setting and have this test report real-time metrics and detailed diagnostics for only those conversations (per interface) that use over a configured percentage of bandwidth resources. For this, set Filter By to Bandwidth Used and configure a bandwidth usage limit (in %) against Minimum Flow Percent. This way, administrators can focus on only those conversations that generate bandwidth-intensive traffic.

Exclude Sources

Provide a comma-separated list of sources that you want excluded from monitoring. For example, your specification can be, 192.168.10.45, 192.168.10.71,192.168.10.220. You can even provide patterns of source IP addresses as a comma-separated list - for instance, 192.168.10.*,192.168.8.1*1, *.168.9.45. Alternatively, you can provide a range of source IP addresses. For example, 192.168.10.32-192.168.10.45.

Note:

  • Sources can be represented using IP addresses only and not host names.
  • All conversations originating from the sources specified against Exclude Sources will be ignored during monitoring.

Exclude Destinations

Provide a comma-separated list of destinations that you want excluded from monitoring. For example, your specification can be, 192.168.10.45, 192.168.10.71,192.168.10.220. You can even provide patterns of destination IP addresses as a comma-separated list - for instance, 192.168.10.*,192.168.8.1*1, *.168.9.45. Alternatively, you can provide a range of source IP addresses. For example, 192.168.10.32-192.168.10.45.

Note:

  • Destinations can be represented using IP addresses only and not host names.
  • All conversations to the destinations specified against Exclude Destinations will be ignored during monitoring.

Exclude Interfaces

Provide a comma-separated list of interfaces that you want excluded from monitoring. For example, your specification can be, FastEtherNet0/0, FastEthernet0/1,FastEtherNet0/2. You can even provide patterns of interfaces as a comma-separated list - for instance, *ethernet*,Fast*.

Show Host Names

This test captures statistics on traffic to/from applications, where each application is by default represented by its IP address in the eG monitoring console. Accordingly, this flag is set to No by default. You can set this flag to Yes so that the names of the individual hosts are displayed in the eG monitoring console instead of their IP addresses.

Ignore Local Traffic

By default, this flag is set to Yes, indicating that the test will ignore the conversations over the intranet on the target NetFlow device. If you want the test to report metrics pertaining to the conversations over the intranet as well, set this flag to No.

Process Tainted Packets

Network latencies and processing bottlenecks can sometimes cause netflow records to be transmitted slowly to the NetFlow collector. In such a situation, you can instruct the collector to either process or ignore the delayed NetFlow records. If you want the metrics reported to pertain to current NetFlow records only, then you may choose to ignore the delayed records. In this case, set this flag to No. If you want old NetFlow records to also be considered when identifying top sources and reporting their traffic statistics, then set this flag to Yes. Typically, any NetFlow record that takes 10 minutes or over to reach the NetFlow collector is deemed a tainted/delayed record.

Sampling, Sampling Rate

By default, NetFlow is designed to process all IP packets on an interface. This is why, the Sampling flag is set to No by default. In some environments however, e.g. on Internet backbones, processing all IP packets can be too costly, due to the extra processing required for each packet and large number of simultaneous flows. This is where sampling is useful. In such environments, set the Sampling flag to Yes.

Once this is done, then each interface will process only 1 packet out of a configured number of packets. Specify the number of packets from which this 1 packet should be picked in the Sampling Rate text box. For instance, to pick 1 out of 1000 packets for sampling, set the Sampling Rate to 1000. Where Sampling is enabled, all NetFlow metrics - particularly, metrics on traffic volume - will be adjusted based on the Sampling Rate you specify.

SNMPPort

The port at which the monitored target exposes its SNMP MIB; the default is 161.

SNMPversion

By default, the eG agent supports SNMP version 1. Accordingly, the default selection in the snmpversion list is v1. However, if a different SNMP framework is in use in your environment, say SNMP v2 or v3, then select the corresponding option from this list.

SNMPCommunity

The SNMP community name that the test uses to communicate with the firewall. This parameter is specific to SNMP v1 and v2 only. Therefore, if the SNMPVersion chosen is v3, then this parameter will not appear.

Username

This parameter appears only when v3 is selected as the SNMP. SNMP version 3 (SNMPv3) is an extensible SNMP Framework which supplements the SNMPv2 Framework, by additionally supporting message security, access control, and remote SNMP configuration capabilities. To extract performance statistics from the MIB using the highly secure SNMP v3 protocol, the eG agent has to be configured with the required access privileges – in other words, the eG agent should connect to the MIB using the credentials of a user with access permissions to be MIB. Therefore, specify the name of such a user against the Username parameter.

Context

This parameter appears only when v3 is selected as the SNMPversion. An SNMP context is a collection of management information accessible by an SNMP entity. An item of management information may exist in more than one context and an SNMP entity potentially has access to many contexts. A context is identified by the SNMPEngineID value of the entity hosting the management information (also called a contextEngineID) and a context name that identifies the specific context (also called a contextName). If the username provided is associated with a context name, then the eG agent will be able to poll the MIB and collect metrics only if it is configured with the context name as well. In such cases therefore, specify the context name of the Username in the context text box. By default, this parameter is set to none.

Authpass

Specify the password that corresponds to the above-mentioned Username. This parameter once again appears only if the SNMPversion selected is v3.

Confirm password

Confirm the Authpass by retyping it here.

Authtype

This parameter too appears only if v3 is selected as the snmpversion. From the authtype list box, choose the authentication algorithm using which SNMP v3 converts the specified username and password into a 32-bit format to ensure security of SNMP transactions. You can choose between the following options:

  • MD5 – Message Digest Algorithm
  • SHA – Secure Hash Algorithm

Encryptflag

This flag appears only when v3 is selected as the SNMPversion. By default, the eG agent does not encrypt SNMP requests. Accordingly, this flag is set to No by default. To ensure that SNMP requests sent by the eG agent are encrypted, select the Yes option.

Encrypttype

If this Encryptflag is set to Yes, then you will have to mention the encryption type by selecting an option from the Encrypttype list. SNMP v3 supports the following encryption types:

  • DES – Data Encryption Standard
  • AES – Advanced Encryption Standard

Encryptpassword

Specify the encryption password here.

Confirm Password

Confirm the encryption password by retyping it here.

Timeout

Specify the duration (in seconds) within which the SNMP query executed by this test should time out in this text box. The default is 10 seconds.

Data Over TCP

By default, in an IT environment, all data transmission occurs over UDP. Some environments however, may be specifically configured to offload a fraction of the data traffic – for instance, certain types of data traffic or traffic pertaining to specific components – to other protocols like TCP, so as to prevent UDP overloads. In such environments, you can instruct the eG agent to conduct the SNMP data traffic related to the monitored target over TCP (and not UDP). For this, set this flag to Yes. By default, this flag is set to No.

DD Frequency

Refers to the frequency with which detailed diagnosis measures are to be generated for this test. The default is 1:1. This indicates that, by default, detailed measures will be generated every time this test runs, and also every time the test detects a problem. You can modify this frequency, if you so desire. Also, if you intend to disable the detailed diagnosis capability for this test, you can do so by specifying none against DD frequency.

Detailed Diagnosis

To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

  • The eG manager license should allow the detailed diagnosis capability
  • Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.
Measurements made by the test
Measurement Description Measurement Unit Interpretation

Total data exchange

Indicates the total amount of data transmitted and received over this conversation during the last measurement period.

KB

Compare the value of this measure across conversations to identify which conversation is contributing to the high level of network traffic.

Use the detailed diagnosis of this measure to determine the top netflows (in terms of the volume of data transacted) between the two hosts engaged in this conversation, and the amount of data transacted in bytes and packets in every flow. With the help of this detailed diagnosis, you can quickly compare the top netflows and know which netflow generated the maximum traffic. Once the problem flow is isolated, you can then investigate why traffic was abnormally high on that flow.

Total packets exchanged

Indicates the total number of packets transmitted and received over this conversation during the last measurement period.

Packets

Compare the value of this measure across conversations to identify which conversation is contributing to the high level of network traffic.

Data exchange rate

Indicates the rate at which data was transmitted/received over this conversation.

Kbps

Packets exchange rate

Indicates the rate at which packets were transmitted/received over this conversation.

Kbps

Total traffic by this conversation

Indicates what percentage of the total traffic on this interface was generated by this conversation.

Percent

A value close to 100% for this measure indicates that traffic generated by this conversation is imposing the maximum load on the network.

If users complain of a latent network, you can compare the value of this measure across conversations to accurately identify which two hosts are exchanging a large volume of data over the network and choking it.

Total bandwidth utilization

Indicates the bandwidth utilized by this conversation.

Percent

A value close to 100% is indicative of excessive bandwidth utilization by this conversation.

If users complain of a latent network, you can compare the value of this measure across conversations to accurately identify which two hosts are engaged in a bandwidth-intensive conversation over the network.

In traffic

Indicates what percentage of total incoming traffic on this interface pertains to this conversation.

Percent

Out traffic

Indicates what percentage of total outgoing traffic on this interface pertains to this conversation.

Percent

Ingress bandwidth utilization

Indicates the percentage of bandwidth utilized by traffic coming into this interface during this conversation.

Percent

A value close to 100% is a cause for concern as it implies a potential congestion in incoming traffic on this interface.

Egress bandwidth utilization

Indicates the percentage of bandwidth utilized by traffic going out of this interface during this conversation.

Percent

A value close to 100% is a cause for concern as it implies a potential congestion in outgoing traffic on this interface.

Data received

Indicates the amount of data received by this interface during this conversation.

KB

Compare the value of this measure across conversations to know which conversation is imposing the maximum load on the interface.

Data sent

Indicates the amount of data sent by this interface during this conversation.

KB

Compare the value of this measure across conversations to know which conversation is imposing the maximum load on the interface.

Packets received

Indicates the number of packets sent by this interface as part of this conversation.

Number

Packets sent

Indicates the number of packets received by this interface during this conversation.

Number

Data received rate

Indicates the rate at which data is received by this interface during this conversation.

Kbps

If the value of this measure consistently drops for this conversation, it could indicate an incoming traffic congestion.

Data transmitted rate

Indicates the rate at which data is sent by this interface during this conversation.

Kbps

If the value of this measure consistently drops for this conversation, it could indicate an outgoing traffic congestion.

Packets received rate

Indicates the rate at which packets are received by this interface during this conversation.

Packets/Sec

If the value of this measure consistently drops for this conversation, it could indicate an incoming traffic congestion.

Packets transmitted rate

Indicates the rate at which packets are transmitted by this interface during this conversation.

Packets/Sec

If the value of this measure consistently drops for this conversation, it could indicate an outgoing traffic congestion.

Protocol

Indicates the protocol used for this conversation.

The values that this measure can report and their corresponding numeric values are listed in the table below:

Measure Value Numeric Value
ICMP 1
IGMP 2
CGP 3
IPv4 4
ST 5
TCP 6
CBT 7
EGP 8
IGP 9
BBN-RCC-MON 10
NVP-II 11
PUP 12
ARGUS 13
EMCON 14
XNET 15
CHAOS 16
UDP 17
MUX 18
RDP 27
IPv6 41
IPv6-Route 43
IPv6-Frag 44
IDRP 45
RSVP 46
Encrypt 47
SWIPE 53
MOBILE 55
IPv6-ICMP 58
IPv6-NoNxt 59
IPv6-Opts 60
VISA 70
PVP 75
DGP 86
IPIP 94
PNNI 102
UDPLite 136

Note:

By default, this test reports the Measure Values listed above to indicate the protocol used by this conversation. In the graph of this measure however, the same is represented using the numeric equivalents only.