Mongo Journaling Statistics Test

With MMAPv1, when a write operation occurs, MongoDB updates the in-memory view. With journaling enabled, MongoDB writes the in-memory changes first to on-disk journal files. If MongoDB should terminate or encounter an error before committing the changes to the data files, MongoDB can use the journal files to apply the write operation to the data files and maintain a consistent state.

With journaling, MongoDB’s storage layer has two internal views of the data set: the private view, used to write to the journal files, and the shared view, used to write to the data files:

1. MongoDB first applies write operations to the private view.
2. MongoDB then applies the changes in the private view to the on-disk journal files in the journal directory roughly every 100 milliseconds. MongoDB records the write operations to the on-disk journal files in batches called group commits. Grouping the commits help minimize the performance impact of journaling since these commits must block all writers during the commit. Writes to the journal are atomic, ensuring the consistency of the on-disk journal files.
3. Upon a journal commit, MongoDB applies the changes from the journal to the shared view.
4. Finally, MongoDB applies the changes in the shared view to the data files. More precisely, at default intervals of 60 seconds, MongoDB asks the operating system to flush the shared view to the data files.

If the mongod instance were to crash without having applied the writes to the data files, the journal could replay the writes to the shared view for eventual write to the data files.

However, if MongoDB takes too long to write changes to the journal, then, in the event that a mongod instance crashes, not all the changes will be available in the journal to enable a complete recovery. In such situations therefore, data loss is inevitable. This is why, it is imperative that administrators detect potential delays in journaling, isolate its root-cause, and resolve the bottleneck before it impacts data recovery. This is where the Mongo Journaling Statistics test helps!

This test monitors writes to the journals and data files, reports the time taken for these writes, and proactively alerts administrators to delays in the journaling process. Using this test, administrators can also figure out at what step of the process the delay may have occurred - when writing to the journal? when committing the writes? or when writing to the data files? This way, the test enables administrators to accurately diagnose the root-cause of journaling slowness, so that they can fix it before it results in data loss.

Note:

This test reports metrics only for those MongoDB instances that use the MMAPv1 storage engine.

Target of the test : A MongoDB server

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for the Mongo database server being monitored.

Configurable parameters for the test

Parameter	Description
Test period	How often should the test be executed.
Host	The host for which the test is to be configured.
Port	The port number at which the specified host listens
Database Name	The test connects to a specific Mongo database to run API commands and pull metrics of interest. Specify the name of this database here. The default value of this parameter is admin.
Username and Password	The eG agent has to be configured with the credentials of a user who has the required privileges to monitor the target MongoDB instance, if the MongoDB instance is access control enabled. To know how to create such a user, refer to How to monitor access control enabled MongoDB database?. If the target MongoDB instance is not access control enabled, then, specify none against the Username and Password parameters.
Confirm Password	Confirm the password by retyping it here.
Authentication Mechanism	Typically, the MongoDB supports multiple authentication mechanisms that users can use to verify their identity. In environments where multiple authentication mechanisms are used, this test enables the users to select the authentication mechanism of their interest using this list box. By default, this is set to None. However, you can modify this settings as per the requirement.
SSL	By default, the SSL flag is set to No, indicating that the target MongoDB server is not SSL-enabled by default. To enable the test to connect to an SSL-enabled MongoDB server, set the SSL flag to Yes.
CA File	A certificate authority (CA) file contains root and intermediate certificates that are electronically signed to affirm that a public key belongs to the owner named in the certificate. If you are looking to monitor the certificates contained within a CA file, then provide the full path to this file in the CA File text box. For example, the location of this file may be: C:\cert\rootCA.pem. If you do not want to monitor the certificates in a CA file, set this parameter to none.
Certificate Key File	A Certificate Key File specifies the path on the server where your private key is stored. If you are looking to monitor the Certificate Key File, then provide the full path to this file in the Certificate Key File text box. For example, the location of this file may be: C:\cert\mongodb.pem. If you do not want to monitor the certificates in a CA file, set this parameter to none.

Measurements made by the test

Measurement	Description	Measurement Unit	Interpretation
Number of commits	Indicates the number of transactions written to the journal during the last time a group commit was performed on the journal.	Number
Data writes to journal	Indicates the amount of data written to the journal during the last time a group commit was performed on the journal.	MB
Preparing time to write to the journal	Indicates the amount of time spent preparing to write to the journal.	Millisecs	Smaller values indicate better journal performance.
Average write time to journal	The amount of time, in milliseconds, spent actually writing to the journal.	Millisecs	Ideally, the value of this measure should be low. A consistent increase in this value is a cause for concern, as it indicates a bottleneck when writing to the journal. File system speeds and device interfaces increase the value of this measure.
Compression ratio of written data to journal	Indicates the compression ratio of the data written to the journal	Percent	A high value is desired for this measure.
Data writes to data file	Indicates the amount of data written from journal to data files during the last group commit.	MB
Average write time to data file	Indicates the amount of time spent writing to data files after journaling.	Millsecs	Ideally, the value of this measure should be low. A consistent increase in this value is a cause for concern, as it indicates a bottleneck when writing to the data files. File system speeds and device interfaces increase the value of this measure.
Early commits	Indicates the number of times MongoDB requested a commit before the scheduled journal group commit interval.	Number	For the MMAPv1 storage engine, you can set the group commit interval using the --journalCommitInterval command line option. The allowed range is 2 to 300 milliseconds. Lower values increase the durability of the journal at the expense of disk performance. Use the value of this measure ensure that your journal group commit interval is not too long for your deployment.
Average time spent for a commit	Indicates the amount of time spent for commits.	Millisecs
Number of commits during write lock	Indicates the count of the commits that occurred while a write lock was held.	Number	Commits in a write lock indicate a MongoDB node under a heavy write load and call for further diagnosis.
Average commits time during write lock	Indicates the amount of time spent for commits that occurred while a write lock was held.	Millisecs