Key Sites Test

Studying network traffic to popular / frequently accessed web sites and measuring the bandwidth usage of this traffic is key to fine-tuning firewall policies in an enterprise and understanding the real bandwidth requirement. The Key Sites test simplifies this! For each web site that is configured for monitoring, this test reports the amount of data transmitted and received and the bandwidth utilized by that site. Web sites that consistently invite heavy traffic and consume excessive bandwidth can be identified in the process. If such sites are mission-critical business sites/applications, then this information will help you to determine the bandwidth required to ensure the peak performance of the sites and thus enable you to right-size your network. If such sites are inconsequential to your business, then this information will point you to where bandwidth is spent unnecessarily; this in turn will prompt you to initiate measures to control/regulate accesses to such sites.

To configure the web sites that this test should monitor, do the following:

  1. Edit the eg_netflow.ini file in the <EG_AGENT_INSTALL_DIR>\agent\config folder (on Windows; on Unix installations of the eG agent, you will find this file in the /opt/egurkha/agent/config folder).
  2. In the TOP SITES section of the file, create a sub-section for the managed NetFlow device. The IP address of the target NetFlow device should be the title of that sub-section.The sub-section title should be specified in square brackets. For instance, if you have managed the NetFlow device using the IP address 192.168.10.25 in your IT infrastructure, then, the specification in the eg_netflow.ini file will be:

    =================================

    TOP SITES

    =================================

    [192.168.10.25]

  3. In this sub-section, specify the URL of the web sites to be monitored, one after another. Against each site URL, specify a comma-separated list of IP addresses of that web site. For example, if the web site www.xyz.com is associated with the IP addresses, 192.168.10.30, 192.168.10.35, 192.168.10.40, 192.168.10.90, then your specification will be as follows:

    =================================

    TOP SITES

    =================================

    [192.168.10.25]

    www.xyz.com=192.168.10.30,192.168.10.35,192.168.10.40,192.168.10.90

    Where a site is associated with a specific range of IP addresses, you can even specify the IP range against the site URL, as shown below:

    =================================

    TOP SITES

    =================================

    [192.168.10.25]

    www.xyz.com=192.168.10.25-192.168.10.45

  4. Likewise, for a NetFlow device, you can configure multiple site URL specifications. For example:

    =================================

    TOP SITES

    =================================

    [192.168.10.25]

    www.xyz.com=192.168.10.30-192.168.10.45

    www.abc.com=192.168.10.125,192.168.10.121,192.168.10.130,192.168.10.90

  5. If a single eG agent is monitoring multiple NetFlow devices, then in the eg_netflow.ini file of that eG agent, you can create multiple sub-sections - one each for every NetFlow device - and configure web sites to be monitored for each device. For example:

    =================================

    TOP SITES

    =================================

    [192.168.10.25]

    www.xyz.com=192.168.10.30-192.168.10.45

    www.abc.com=192.168.10.125,192.168.10.121,192.168.10.130,192.168.10.90

    [192.168.10.200]

    www.eazycart.com=192.168.10.1,192.168.10.2

    www.fb.com=192.168.10.5,192.168.10.9

  6. Finally, save the file.

Target of the test : A Netflow Device

Agent deploying the test : An external agent

Outputs of the test : One set of results for every site URL that is configured for monitoring in the eg_netflow.ini file

Configurable parameters for the test
Parameter Description

Test period

How often should the test be executed

Host

The host for which the test is to be configured.

Process Tainted Packets

Network latencies and processing bottlenecks can sometimes cause netflow records to be transmitted slowly to the NetFlow collector. In such a situation, you can instruct the collector to either process or ignore the delayed NetFlow records. If you want the metrics reported to pertain to current NetFlow records only, then you may choose to ignore the delayed records. In this case, set this flag to No. If you want old NetFlow records to also be considered when reporting traffic statistics, then set this flag to Yes. Typically, any NetFlow record that takes 10 minutes or over to reach the NetFlow collector is deemed a tainted/delayed record.

Sampling, Sampling Rate

By default, NetFlow is designed to process all IP packets on an interface. This is why, the Sampling flag is set to No by default. In some environments however, e.g. on Internet backbones, processing all IP packets can be too costly, due to the extra processing required for each packet and large number of simultaneous flows. This is where sampling is useful. In such environments, set the Sampling flag to Yes.

Once this is done, then each interface will process only 1 packet out of a configured number of packets. Specify the number of packets from which this 1 packet should be picked in the Sampling Rate text box. For instance, to pick 1 out of 1000 packets for sampling, set the Sampling Rate to 1000. Where Sampling is enabled, all NetFlow metrics - particularly, metrics on traffic volume - will be adjusted based on the Sampling Rate you specify.

DD Frequency

Refers to the frequency with which detailed diagnosis measures are to be generated for this test. The default is 1:1. This indicates that, by default, detailed measures will be generated every time this test runs, and also every time the test detects a problem. You can modify this frequency, if you so desire. Also, if you intend to disable the detailed diagnosis capability for this test, you can do so by specifying none against DD frequency.

Detailed Diagnosis

To make diagnosis more efficient and accurate, the eG Enterprise suite embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

  • The eG manager license should allow the detailed diagnosis capability
  • Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.
Measurements made by the test
Measurement Description Measurement Unit Interpretation

Total data exchange

Indicates the total amount of data transmitted and received by this web site during the last measurement period.

KB

Compare the value of this measure across web sites to identify which web site is contributing to the high level of network traffic.

Use the detailed diagnosis of this measure to determine the top netflows (in terms of the volume of data transacted) to or from this web site, and the amount of data transacted in bytes and packets in every flow. With the help of this detailed diagnosis, you can quickly compare the top netflows, know which netflow generated the maximum traffic, and figure out which source traffic originated from. Once the problem source is isolated, you can then investigate why traffic from that source is high.

Total packets exchanged

Indicates the total number of packets transmitted and received by this web site during the last measurement period.

Packets

Compare the value of this measure across sources to identify which web site is contributing to the high level of network traffic

Data exchange rate

Indicates the rate at which this web site transmitted/received data.

Kbps

Packets exchange rate

Indicates the rate at which this web site transmitted/received packets.

Kbps

Total traffic to this web site

Indicates what percentage of the total traffic on this interface was to this web site.

Percent

A value close to 100% for this measure indicates that traffic to this web site is imposing the maximum load on the network. If users complain of a latent network, you can compare the value of this measure across web sites to accurately identify the web site that is responsible for any congestion on the network.

Total bandwidth utilization

Indicates the bandwidth utilized by this web site.

Percent

A value close to 100% is indicative of excessive bandwidth utilization by this web site when receiving data.

If users complain of a latent network, you can compare the value of this measure across web sites to accurately identify the web site that is responsible for any congestion on the network.

In traffic

Indicates what percentage of total incoming traffic on this interface pertains to this web site.

Percent

Out traffic

Indicates what percentage of total outgoing traffic on this interface pertains to this web site.

Percent

Ingress bandwidth utilization

Indicates the percentage of bandwidth utilized by traffic coming into this interface when receiving data for this web site from different sources and when receiving data from this web site.

Percent

A value close to 100% is a cause for concern as it implies a potential congestion in incoming traffic on this interface.

Egress bandwidth utilization

Indicates the percentage of bandwidth utilized by traffic going out of this interface when transmitting data to this web site and when transmitting data for this web site to a source.

Percent

A value close to 100% is a cause for concern as it implies a potential congestion in outgoing traffic on this interface.

Data received

Indicates the amount of data received by this web site.

KB

Compare the value of this measure across web sites to know which web site is receiving maximum data over this interface.

Data sent

Indicates the amount of data sent by this web site.

KB

Compare the value of this measure across web sites to know which web site is transmitting maximum data over this interface.

Packets received

Indicates the number of packets sent by this web site.

Number

Packets sent

Indicates the number of packets received by this web site.

Number

Data received rate

Indicates the rate at which data is received by this web site.

Kbps

If the value of this measure consistently drops for this web site, it could indicate an incoming traffic congestion.

Data transmitted rate

Indicates the rate at which data is sent by this web site.

Kbps

If the value of this measure consistently drops for this web site, it could indicate an outgoing traffic congestion.

Packets received rate

Indicates the rate at which packets are received by this web site.

Packets/Sec

If the value of this measure consistently drops for this web site, it could indicate an incoming traffic congestion.

Packets transmitted rate

Indicates the rate at which packets are transmitted by this web site.

Packets/Sec

If the value of this measure consistently drops for this web site, it could indicate an outgoing traffic congestion.