TCP Port Test

This operating system-specific test periodically tracks the status of TCP connections to and from a server. This test can be used in different ways. For instance, an administrator can use this test to determine the number of connections that currently exist to a specific TCP port on the server (e.g., the web server port). Alternately, the administrator can also determine the number of TCP connections established from one server to another server - for example, from a web server to a specific application server.

To understand the measures reported by this test, consider the state/transition diagram for the TCP protocol below. The different TCP protocol states are in round-ended boxes, and the transitions are the labeled arrows. The transitions show how your program can make TCP move from one state to another. It also shows how the remote peer can make your stack change TCP states, and how you can recognize these changes at the application level.

A TCP client calls the connect() function (or similar), which causes TCP to send an empty packet with the SYN control bit set (SYN_SENT). The remote peer’s stack sees this “synchronize” request, and sends back an empty packet with the SYN and ACK bits set (i.e. “I acknowledge your synchronize request”). When the client receives the SYN/ACK packet, it sends back an ACK packet, and reports a successful connection to the client program.

The TIME_WAIT state is a safety mechanism, to catch stray packets for that connection after the connection is “officially” closed.

FIN_WAIT_1 usually happens when the local program calls shutdown() with the “how” parameter set to 1 or SD_SEND, but the remote peer doesn’t respond. Likewise FIN_WAIT_2 happens when the remote peer shuts down its sending half of the connection, and your program doesn’t respond. Since FIN_WAIT states often last up to 10 minutes, it’s well worth the effort to fix the problem that’s causing these FIN_WAIT states.

To use the TcpPortTest, an administrator should specify a list of TCP source/port or destination/port combinations that he/she is interested in monitoring. For each such combination, the TcpPortTest reports the number of TCP connections in each of the TCP protocol states. Analysis of the results can point to scenarios that need attention - e.g., abnormally high established connections to a specific TCP port, unusually large number of connections in the FIN_WAIT state, etc. Often in multi-tier infrastructures that include a number of inter-dependent application tiers, it is also interesting to compare the number of connections established to each tier and correlate the increase/decrease of connections across tiers.

Target of the test : An application being monitored

Agent deploying the test : An internal agent

Outputs of the test : One set of results for every pattern

Configurable parameters for the test
  1. TEST PERIOD - How often should the test be executed
  2. HOST - The host for which the test is to be configured
  3. PORTNO - The port number on which the application being monitored is running
  4. targetappports - This parameter defines the source/port and/or destination/port combinations to be monitored. This parameter is specified in the format:

    PatternName:LocalIP:LocalPort@RemoteIP:RemotePort

    PatternName is a unique name by which the pattern being defined is to be identified. This is the name that would appear alongside the test name in the monitor interface.

    When configuring this parameter, decide whether you are monitoring connections to the application or connections from the application to other applications. If the connections to an application running on the local system are to be monitored, the LocalIp and LocalPort become relevant. For example, if the number of connections to a web server on the local system have to be monitored, the LocalIp can be “*” (indicating that all the local IP addresses are to be considered), and the LocalPort can be “80”, to monitor the web server running on port 80. On the other hand, if the web server is running on a specific IP address, specify this IP address in the LocalIp field.

    The RemoteIP is the IP address of the remote end of the TCP connection. In the example above, TCP connections can be established from any remote address to the web server. Hence, the RemoteIP should be “*” in this example. Likewise, the RemotePort is the TCP port being used to connect to the application being monitored. In the example above, clients can use any TCP port to connect to the application, and hence, the RemotePort setting should be “*”. 

    To conclude, to monitor all the connections to a web server running on port 80 and configured to use an IP address 192.168.10.8, the TARGETAPPPORTS specification should be WebUsage:192.168.10.8:80@*:*.

    Suppose the administrator also wants to monitor the TCP connections going out of the web server to a J2EE application server that is listening on IP address 192.168.10.20 on port 6010, then the corresponding TARGETAPPPORTS configuration should be J2EE:*:*@192.168.10.20:6010. This indicates that the clients can be using any IP address/port to connect to the application server.

    The complete tartgetappports specification for this example, will hence be:

    WebUsage:192.168.10.8:80@*:*, J2EE:*:*@192.168.10.20:6010

    As another example, you can also instruct the eG Enterprise system to monitor all TCP connections from IP addresses in the range 192.168.10.30-192.168.10.39, to IP addresses in the range 192.168.10.40-192.168.10.49. The pattern specification for this would be:

    Connection34:192.168.10.3*:*@192.168.10.4*:*

Measurements made by the test
Measurement Description Measurement Unit Interpretation

Syn-sent connections:

Indicates the number of connections that are in the process of being established by the host to other server(s)

Number

 

Syn-received connections:

Indicates the number of connections that are in the process of being established by remote hosts to this host

Number

 

Established connections:

Indicates the total number of TCP connections on this host for the port number(s) specified in the test arguments

Number

The number of TCP connections established to a server is one indicator of the server workload

 

Close-wait connections:

Indicates the current number of TCP connections to a port that are in the TCP CLOSE_WAIT state. Connections remain in the close wait state when they are waiting for a process to close the TCP socket.

Number

 

Fin-wait-1 connections:

Indicates the number of TCP connections to a TCP port that are in the FIN_WAIT_1 state. A TCP connection moves to the FIN_WAIT_1 state when a local program closes a socket but the remote server does not respond.

Number

A large number of FIN_WAIT_1 connections can occur if clients are not properly closing down TCP connections. A connection may linger in this state for tens of minutes.

 

Fin-wait-2 connections:

Indicates the number of TCP connections to a TCP port that are in the FIN_WAIT_2 state. A connection moves to the FIN_WAIT_2 state when a remote server shuts down its side of a TCP connection and the local server does not respond to it.

Number

 

Time-wait connections:

Indicates the number of connections in the TCP TIME_WAIT state. The TIME_WAIT state is a safety mechanism, to catch stray packets for that connection after the connection is “officially” closed. Since the maximum time that such stray packets can exist is 2 times the maximum round-trip time, the TIME_WAIT state lasts twice the round-trip period. Roughly, the duration is 30-120 seconds.

Number

 

TCP send queue:

Send-Q is used to show the socket buffer status. This indicates the number of bytes that have been sent to the destination, and are awaiting acknowledgment.

(Available only for Solaris, Linux, HP-UX and AIX)

Bytes/sec

A high value of this measure indicates a poor network response.

TCP receive queue:

Receive-Q is used to show the socket buffer status. The number indicates the number of bytes received from the source and copied.

(Available only for Solaris, Linux, HP-UX and AIX)

Bytes/sec

A high value of this measure indicates a poor network response.