Mongo Journaling Statistics Test

With MMAPv1, when a write operation occurs, MongoDB updates the in-memory view. With journaling enabled, MongoDB writes the in-memory changes first to on-disk journal files. If MongoDB should terminate or encounter an error before committing the changes to the data files, MongoDB can use the journal files to apply the write operation to the data files and maintain a consistent state.

With journaling, MongoDB’s storage layer has two internal views of the data set: the private view, used to write to the journal files, and the shared view, used to write to the data files:

MongoDB first applies write operations to the private view.
MongoDB then applies the changes in the private view to the on-disk journal files in the journal directory roughly every 100 milliseconds. MongoDB records the write operations to the on-disk journal files in batches called group commits. Grouping the commits help minimize the performance impact of journaling since these commits must block all writers during the commit. Writes to the journal are atomic, ensuring the consistency of the on-disk journal files.
Upon a journal commit, MongoDB applies the changes from the journal to the shared view.
Finally, MongoDB applies the changes in the shared view to the data files. More precisely, at default intervals of 60 seconds, MongoDB asks the operating system to flush the shared view to the data files.

If the mongod instance were to crash without having applied the writes to the data files, the journal could replay the writes to the shared view for eventual write to the data files.

However, if MongoDB takes too long to write changes to the journal, then, in the event that a mongod instance crashes, not all the changes will be available in the journal to enable a complete recovery. In such situations therefore, data loss is inevitable. This is why, it is imperative that administrators detect potential delays in journaling, isolate its root-cause, and resolve the bottleneck before it impacts data recovery. This is where the Mongo Journaling Statistics test helps!

This test monitors writes to the journals and data files, reports the time taken for these writes, and proactively alerts administrators to delays in the journaling process. Using this test, administrators can also figure out at what step of the process the delay may have occurred - when writing to the journal? when committing the writes? or when writing to the data files? This way, the test enables administrators to accurately diagnose the root-cause of journaling slowness, so that they can fix it before it results in data loss.

Note:

This test reports metrics only for those MongoDB instances that use the MMAPv1 storage engine.

Target of the test : A MongoDB server

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for the Mongo database server being monitored.

Configurable parameters for the test
Parameter	Description
Test period	How often should the test be executed.
Host	The host for which the test is to be configured.
Port	The port number at which the specified host listens
Database Name	The test connects to a specific Mongo database to run API commands and pull metrics of interest. Specify the name of this database here. The default value of this parameter is admin.
Username and Password	The eG agent has to be configured with the credentials of a user who has the required privileges to monitor the target MongoDB instance, if the MongoDB instance is access control enabled. To know how to create such a user, refer to How to monitor access control enabled MongoDB database? . If the target MongoDB instance is not access control enabled, then, specify none against the Username and Password parameters.
Confirm Password	Confirm the password by retyping it here.
Authentication Mechanism	Typically, the MongoDB supports multiple authentication mechanisms that users can use to verify their identity. In environments where multiple authentication mechanisms are used, this test enables the users to select the authentication mechanism of their interest using this list box. By default, this is set to None. However, you can modify this settings as per the requirement.
SSL	By default, the SSL flag is set to No, indicating that the target MongoDB server is not SSL-enabled by default. To enable the test to connect to an SSL-enabled MongoDB server, set the SSL flag to Yes.
CA File	A certificate authority (CA) file contains root and intermediate certificates that are electronically signed to affirm that a public key belongs to the owner named in the certificate. If you are looking to monitor the certificates contained within a CA file, then provide the full path to this file in the CA File text box. For example, the location of this file may be: C:\cert\rootCA.pem. If you do not want to monitor the certificates in a CA file, set this parameter to none.
Certificate Key File	A Certificate Key File specifies the path on the server where your private key is stored. If you are looking to monitor the Certificate Key File, then provide the full path to this file in the Certificate Key File text box. For example, the location of this file may be: C:\cert\mongodb.pem. If you do not want to monitor the certificates in a CA file, set this parameter to none.
CA PEM File	A .pem file is a container that may just include the public certificate or the entire certificate chain (private key, public key and root certificates). If the connection requires server authentication and the server certificate is in the .pem format, then, the target instance presents the CA PEM File that contains the server certificate to its clients to establish the instance's identity. Therefore, you should specify the full path to the CA PEM file available in the target MongoDB server in the CA PEM File text box. For example, the location of this file may be: C:\app\openSSL\SSLcert\test-ca.pem.
Client PEM File	If the target instance requires a certificate key file that is in .pem format from the client to verify the client's identity, then, to establish a connection with the target server, the eG agent should access the client certificate. For this, specify the full path to the Client PEM file in the Client PEM File text box. For example, the location of this file may be: C:\app\openSSL\SSLcert\test-client.pem.
CA Cert File	This parameter is applicable only if the target MongoDB server is SSL-enabled and CA PEM File parameter is set to none.The certificate file is a public-key certificate following the x.509 standard. It contains information about the identity of the server, such as its name, geolocation, and public key. Essentially, it’s a certificate that the server serves to the connecting users to prove that they are what they claim to be. Therefore, specify the full path to the server root certificate or certificate file that is signed by the CA in .crt file format for the server in the CA Cert File text box. For example, the location of this file may be: C:\app\eGurkha\JRE\lib\security\mongodb-test-ca.crt. By default, this parameter is set to none.
Client Cert File	This parameter is applicable only if the target MongoDB server is SSL-enabled and Client PEM File parameter is set to none.In order to collect metrics from the target MongoDB, the eG agent requires client certificate in .p12 format. Hence, specify the full path to the Client certificate file in .p12 format in the Client Cert File text box. For example, the location of this file may be: C:\app\eGurkha\JRE\lib\security\test-client.p12 . To know how to generate .p12 file from Client PEM file, refer to How to import a Certificate that is in the PEM Format? By default, this parameter is set to none.
Client Cert Password	Provide the password for .p12 Client certificate file in the Client Cert Password text box.
AWS Key ID, AWS Secret Key,Confirm Password	If you are monitoring MongoDB server hosted on the AWS cloud, the eG agent has to be configured with the AWS AccessKey ID and Secret Key to connect with the AWS cloud and collect the required metrics. Therefore, Specify the AWS Key ID and AWS Secret Key and confirm the password by re-typing it in the Confirm Password text box. To obtain the AWS Access key and secret key, refer toObtaining AWS Access Key and Obtaining AWS Secret Key.
Atlas URI	MongoDB Atlas is a NoSQL Database-as-a-Service offering in the public cloud. If the target MongoDB server is deployed and managed in MongoDB Atlas, then the eG agent has to be configured with the MongoDB Atlas connection URI,a unique identifier for connecting to a MongoDB server, in the Atlas URI text box to access the target MongoDB server hosted on Atlas and collect the required metrics.

Measurements made by the test
Measurement	Description	Measurement Unit	Interpretation
Number of commits	Indicates the number of transactions written to the journal during the last time a group commit was performed on the journal.	Number
Data writes to journal	Indicates the amount of data written to the journal during the last time a group commit was performed on the journal.	MB
Preparing time to write to the journal	Indicates the amount of time spent preparing to write to the journal.	Seconds	Smaller values indicate better journal performance.
Average write time to journal	The amount of time spent actually writing to the journal.	Seconds	Ideally, the value of this measure should be low. A consistent increase in this value is a cause for concern, as it indicates a bottleneck when writing to the journal. File system speeds and device interfaces increase the value of this measure.
Compression ratio of written data to journal	Indicates the compression ratio of the data written to the journal	Percent	A high value is desired for this measure.
Data writes to data file	Indicates the amount of data written from journal to data files during the last group commit.	MB
Average write time to data file	Indicates the amount of time spent writing to data files after journaling.	Seconds	Ideally, the value of this measure should be low. A consistent increase in this value is a cause for concern, as it indicates a bottleneck when writing to the data files. File system speeds and device interfaces increase the value of this measure.
Early commits	Indicates the number of times MongoDB requested a commit before the scheduled journal group commit interval.	Number	For the MMAPv1 storage engine, you can set the group commit interval using the --journalCommitInterval command line option. The allowed range is 2 to 300 milliseconds. Lower values increase the durability of the journal at the expense of disk performance. Use the value of this measure ensure that your journal group commit interval is not too long for your deployment.
Average time spent for a commit	Indicates the amount of time spent for commits.	Seconds
Number of commits during write lock	Indicates the count of the commits that occurred while a write lock was held.	Number	Commits in a write lock indicate a MongoDB node under a heavy write load and call for further diagnosis.
Average commits time during write lock	Indicates the amount of time spent for commits that occurred while a write lock was held.	Seconds