Oracle Dead Kill Processes Test

If one/more sessions or processes on the Oracle server are obstructing the execution of a few other sessions/processes, then, it is quiet natural for administrators to want to kill the blocking sessions/processes to ensure the smooth execution of critical database transactions. Typically, these ‘dead’ sessions/processes continue to consume resources, until the PMON process automatically cleans up these sessions/processes. If cleanup is delayed, then the Oracle instance will not be able to release those objects and resources that have been locked by the dead sessions/processes for long time periods. In such situations, administrators often resort to killing these dead sessions/processes at the operating system-level, so as to hasten the release of valuable resources. Before attempting the OS-level kill, administrators should first figure out which sessions/processes are ‘dead’ presently and how long they have been ‘dead’. This can be ascertained using the Oracle Dead Kill Processes test.

This test auto-discovers the dead processes/sessions and reports the current cleanup state of each process/session. In addition, the test reveals the duration for which each process/session remained dead and the count of processes that are being blocked by that dead process/session. This way, administrators can determine whether/not cleanup is occurring as per schedule, and if not, how badly the delay in cleanup is affecting other processes. Alongside, administrators can figure out whether an OS-level process kill is justified or not.

Note:

This test is applicable only for CDB(Container Database) configuration of an Oracle Database with Multi-tenant support.

Target of the test : An Oracle 12c server

Agent deploying the test : An internal agent

Outputs of the test : One set of results for deadprocessaddress_deadsessionaddress on the Oracle instance monitored.

Configurable parameters for the test
Parameter	Description
Test period	How often should the test be executed
Host	The host for which the test is to be configured.
Port	The port on which the server is listening.
Username	In order to monitor an Oracle database server, a special database user account has to be created in every Oracle database instance that requires monitoring. A Click here hyperlink is available in the test configuration page, using which a new oracle database user can be created. Alternatively, you can manually create the special database user. When doing so, ensure that this user is vested with the select_catalog_role and create session privileges. The sample script we recommend for user creation (in Oracle database server versions before 12c) for eG monitoring is: create user oraeg identified by oraeg create role oratest; grant create session to oratest; grant select_catalog_role to oratest; grant oratest to oraeg; The sample script we recommend for user creation (in Oracle database server 12c) for eG monitoring is: alter session set container=<Oracle_service_name>; create user <user_name>identified by <user_password> container=current default tablespace <name_of_default_tablespace> temporary tablespace <name_of_temporary_tablespace>; Grant create session to <user_name>; Grant select_catalog_role to <user_name>; The name of this user has to be specified here.
Password	Specify the password of the specified database user.
Confirm Password	Confirm the Password by retyping it here.
IsPassive	If the value chosen is Yes, then the Oracle server under consideration is a passive server in an Oracle cluster. No alerts will be generated if the server is not running. Measures will be reported as “Not applicable" by the agent if the server is not up.
SSL	By default, this flag is set to No, as the target Oracle database is not SSL-enabled by default. If the target database is SSL-enabled, then set this flag to Yes.
SSL Cipher	This parameter is applicable only if the target Oracle database is SSL-enabled, if not, set this parameter to none. A cipher suite is a set of cryptographic algorithms that are used before a client application and server exchange information over an SSL/TLS connection. It consist of sets of instructions on how to secure a network through SSL (Secure Sockets Layer) or TLS (Transport Layer Security). In this text box, provide a comma-seperated list of cipher suites that are allowed for SSL/TLS connection to the target database. By default, this parameter is set to none.
Truststore File	This parameter is applicable only if the target Oracle database is SSL-enabled, if not, set this parameter to none. TrustStore is used to store certificates from Certified Authorities (CA) that verify and authenticate the certificate presented by the server in an SSL connection. Therefore, the eG agent should have access to the truststore where the certificates are stored to authenticate and connect with the target database and collect metrics. For this, first import the certificates into the following default location <eG_INSTALL_DIR>/lib/security/mytruststore.jks. To know how to import the certificate into the truststore, refer toPre-requisites for monitoring Oracle Cluster. Then, provide the truststore file name in this text box. For example: mytruststore.jks. By default, none is specified against this text box.
Truststore Type	This parameter is applicable only if the target Oracle database is SSL-enabled, if not, set this parameter to none.Specify the type of truststore that contains the certificates for server authentication in this text box. For eg.,JKS. By default, this parameter is set to the value none.
Truststore Password	This parameter is applicable only if the target Oracle database is SSL-enabled, if not, set this parameter to none. If a Truststore File name is provided, then, in this text box, provide the password that is used to obtain the associated certificate details from the Truststore File. By default, this parameter is set to none.
Keystore File	This parameter is applicable only if the target Oracle database is SSL-enabled, if not, set this parameter to none. Keystore contains the private keys for the certificates that the client can provide to the server upon request. eG agent requires access to the keystore where client certificate is stored to send that to the server so that the server validates the certificate against the one contained in its trustore. For this purpose, first create the client certificate in the following default location /opt/egurkha/jre/lib/security/egmqsslstore.jks.
Keystore Password	This parameter is applicable only if the target Oracle database is SSL-enabled, if not, set this parameter to none. If a Keystore File name or file path is provided, then, in this text box, provide the password that is used to obtain the associated certificate details from the Keystore File.
Confirm Password	Confirm the Password for Keystore by retyping it here.

Measurements made by the test

Measurement

Description

Measurement Unit

Interpretation

Process state:

Indicates the current cleanup state of this process.

The values that this measure can report and their corresponding numeric values have been discussed hereunder:

Measure Value	Description	Numeric Value
unsafe to attempt	Occurs for a killed session that has not been moved, so no cleanup can occur on it yet	1
cleanup pending	Occurs for a dead process / killed session that can be cleaned up, but PMON has not yet made an attempt	2
resources freed	Occurs for a dead process / killed session where all children have been freed, but the process / killed session itself is not yet freed	3
resources freed – pending ack	Occurs for a killed session where all children have been freed, but the session itself cannot be freed until the owner has acknowledged it	4
partial cleanup	Occurs if some of the children have been cleaned up	5

Note:

By default, this measure reports the above-mentioned Measure Values while indicating the current cleanup state of a dead process. However, in the graph of this measure, the same will be represented using the corresponding numeric equivalents only.

Dead time:

Indicates how long it has been since this process was marked dead or this session was marked killed.

Secs

A consistent increase in the value of this measure is a cause for concern as it indicates that auto-cleanup has not occurred. This can cause the dead process/session to continue consuming resources and blocking object, thereby degrading server performance.

Number blocked:

Indicates the count of processes that are blocked by this process.

Number

A high value indicates that the dead process is impeding the execution of many other processes, some of which may also be mission-critical.

If the Dead time of such a process is also very high, it is a matter of great concern, and must be looked into immediately.

In such circumstances, you may want to consider killing the process at the OS-level. On a Unix system, you can issue the KILL -9 <PID> command at the Shell prompt to kill the process at that level.