Mongo Global Lock Queue Test

MongoDB uses multi-granular reader-writer locking. Operations are allowed to lock at the global, database or collection level. Individual storage engines are allowed to implement their own concurrency control below the collection level (e.g., at the document-level in WiredTiger).

A global 'instance-wide' lock is required when some global operations - typically, short lived operations involving multiple databases - are performed. The following MongoDB operations lock multiple databases, and may hence require a global lock:

• db.copyDatabase() must lock the entire mongod instance at once.
• db.repairDatabase() obtains a global write lock and will block other operations until it finishes.
• Journaling, which is an internal operation, locks all databases for short intervals. All databases share a single journal.
• User authentication requires a read lock on the admin database for deployments using 2.6 user credentials. For deployments using the 2.4 schema for user credentials, authentication locks the admin database as well as the database the user is accessing.
• All writes to a replica set’s primary lock both the database receiving the writes and then the local database for a short time. The lock for the local database allows the mongod to write to the primary’s oplog and accounts for a small portion of the total time of the operation.

MongoDB uses a readers-writer [ed: also known as “multi-reader” or “shared exclusive”] lock that allows concurrent reads access, but gives exclusive access to a single write operation. When a read lock exists, many read operations may use this lock. However, when a write lock exists, a single write operation holds the lock exclusively, and no other read or write operations may share the lock. If read or write requests are received by a database instance on which a write lock pre-exists, such requests will be placed in a queue, where they will wait until the write lock is released. While some level of queuing is to be expected, an abnormal number of enqueued requests is a cause for concern, as it only means that application requests are being continuously blocked; if the problem persists, it can cause application performance to suffer greatly. It is therefore imperative that requests waiting in queues for locks are tracked and consistent increase in this request count brought to the attention of administrators. This is what the Mongo Global Lock Queue test does.

This test monitors the current global lock queue of a Mongo database server and reports the total count of requests in the queue. Additionally, the test also reports how many of these enqueued requests are read requests and how many are write requests. This way, the test indicates whether/not locks are released and granted to new requests in a timely manner. Locking inefficiencies are thus revealed. In addition, the test also indicates what type of requests the server is blocking more - read requests? or write requests? From this, administrators can figure out what type of lock currently exists on the server being monitored - read lock? or write lock?

Target of the test : A MongoDB server

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for the Mongo database server being monitored.

Configurable parameters for the test

Parameter	Description
Test period	How often should the test be executed.
Host	The host for which the test is to be configured.
Port	The port number at which the specified host listens.
Database Name	The test connects to a specific Mongo database to run API commands and pull metrics of interest. Specify the name of this database here. The default value of this parameter is admin.
Username and Password	The eG agent has to be configured with the credentials of a user who has the required privileges to monitor the target MongoDB instance, if the MongoDB instance is access control enabled. To know how to create such a user, refer to How to monitor access control enabled MongoDB database?. If the target MongoDB instance is not access control enabled, then, specify none against the Username and Password parameters.
Confirm Password	Confirm the password by retyping it here.
Authentication Mechanism	Typically, the MongoDB supports multiple authentication mechanisms that users can use to verify their identity. In environments where multiple authentication mechanisms are used, this test enables the users to select the authentication mechanism of their interest using this list box. By default, this is set to None. However, you can modify this settings as per the requirement.
SSL	By default, the SSL flag is set to No, indicating that the target MongoDB server is not SSL-enabled by default. To enable the test to connect to an SSL-enabled MongoDB server, set the SSL flag to Yes.
CA File	A certificate authority (CA) file contains root and intermediate certificates that are electronically signed to affirm that a public key belongs to the owner named in the certificate. If you are looking to monitor the certificates contained within a CA file, then provide the full path to this file in the CA File text box. For example, the location of this file may be: C:\cert\rootCA.pem. If you do not want to monitor the certificates in a CA file, set this parameter to none.
Certificate Key File	A Certificate Key File specifies the path on the server where your private key is stored. If you are looking to monitor the Certificate Key File, then provide the full path to this file in the Certificate Key File text box. For example, the location of this file may be: C:\cert\mongodb.pem. If you do not want to monitor the certificates in a CA file, set this parameter to none.

Measurements made by the test

Measurement	Description	Measurement Unit	Interpretation
Total lock in current queue	Indicates the number of requests currently in queue.	Number	A consistently small queue, particularly of shorter operations, should cause no concern. However, if the value of this measure increases consistently, it could mean that new locks are not being granted to the Mongo database server being monitored. This can happen if an operation has been holding a write lock for a long time. Further investigation will reveal which operation this is and why it is not releasing the lock.
Read lock in current queue	Indicates the number of read requests currently in queue.	Number	If the Total locks in queue measure is exhibiting abnormal trends, observing the behavior of these two measures will provide useful pointers to what could be causing the high queue length. For instance, if the value of the Read locks in current queue measure and the Write lock in current queue measure is increasing steadily over time, it could mean that an operation is probably holding a write lock for a long time. If there are more Read locks in current queue, but very less Write locks in current queue, it could mean that write requests are being prioritized over read requests and are being granted locks.
Write lock in current queue	Indicates the number of write requests currently in queue.	Number