Mongo Background Flushing Test

By default, MongoDB instances using the MMAPv1 storage engine write in-memory changes to disk every 60 seconds. If Journaling is enabled, then the MongoDB server writes changes to the on-disk journal first. These changes will then flush from journal to the disk. If the server crashes before all the changes are flushed to the disk, then journaling ensures that the changes can still be recovered from the journal. However, where Journaling is not enabled, if changes in memory are not flushed to the disk quickly, then the sudden terminaton of the MongoDB server can result in loss of critical data. If this is to be avoided, then administrators should be able to proactively detect any potential slowness in background flushing and promptly initiate measures to pre-empt the data loss than an server crash can cause. The Mongo Background Flushing test helps with this.

This test tracks flushes to disk and reports the average time taken by the monitored server to flush writes in memory to disk. In the process, the test proactively alerts administrators to slowness in disk writes. The test also reports the duration of the last disk write, thus enabling administrators to figure out when the slowness could have occurred - did it creep in recently? or has it been persistent?

Note:

This test reports metrics only for those MongoDB instances that use the MMAPv1 storage engine.

Target of the test : A MongoDB server

Agent deploying the test : An internal/remote agent

Outputs of the test : One set of results for the Mongo database server being monitored

Configurable parameters for the test
Parameter	Description
Test period	How often should the test be executed
Host	The host for which the test is to be configured.
Port	The port number at which the specified host listens
Database Name	The test connects to a specific Mongo database to run API commands and pull metrics of interest. Specify the name of this database here. The default value of this parameter is admin.
Username and Password	The eG agent has to be configured with the credentials of a user who has the required privileges to monitor the target MongoDB instance, if the MongoDB instance is access control enabled. To know how to create such a user, refer to How to monitor access control enabled MongoDB database? . If the target MongoDB instance is not access control enabled, then, specify none against the Username and Password parameters.
Confirm Password	Confirm the password by retyping it here.
Authentication Mechanism	Typically, the MongoDB supports multiple authentication mechanisms that users can use to verify their identity. In environments where multiple authentication mechanisms are used, this test enables the users to select the authentication mechanism of their interest using this list box. By default, this is set to None. However, you can modify this settings as per the requirement.
SSL	By default, the SSL flag is set to No, indicating that the target MongoDB server is not SSL-enabled by default. To enable the test to connect to an SSL-enabled MongoDB server, set the SSL flag to Yes.
CA File	A certificate authority (CA) file contains root and intermediate certificates that are electronically signed to affirm that a public key belongs to the owner named in the certificate. If you are looking to monitor the certificates contained within a CA file, then provide the full path to this file in the CA File text box. For example, the location of this file may be: C:\cert\rootCA.pem. If you do not want to monitor the certificates in a CA file, set this parameter to none.
Certificate Key File	A Certificate Key File specifies the path on the server where your private key is stored. If you are looking to monitor the Certificate Key File, then provide the full path to this file in the Certificate Key File text box. For example, the location of this file may be: C:\cert\mongodb.pem. If you do not want to monitor the certificates in a CA file, set this parameter to none.
CA PEM File	A .pem file is a container that may just include the public certificate or the entire certificate chain (private key, public key and root certificates). If the connection requires server authentication and the server certificate is in the .pem format, then, the target instance presents the CA PEM File that contains the server certificate to its clients to establish the instance's identity. Therefore, you should specify the full path to the CA PEM file available in the target MongoDB server in the CA PEM File text box. For example, the location of this file may be: C:\app\openSSL\SSLcert\test-ca.pem.
Client PEM File	If the target instance requires a certificate key file that is in .pem format from the client to verify the client's identity, then, to establish a connection with the target server, the eG agent should access the client certificate. For this, specify the full path to the Client PEM file in the Client PEM File text box. For example, the location of this file may be: C:\app\openSSL\SSLcert\test-client.pem.
CA Cert File	This parameter is applicable only if the target MongoDB server is SSL-enabled and CA PEM File parameter is set to none.The certificate file is a public-key certificate following the x.509 standard. It contains information about the identity of the server, such as its name, geolocation, and public key. Essentially, it’s a certificate that the server serves to the connecting users to prove that they are what they claim to be. Therefore, specify the full path to the server root certificate or certificate file that is signed by the CA in .crt file format for the server in the CA Cert File text box. For example, the location of this file may be: C:\app\eGurkha\JRE\lib\security\mongodb-test-ca.crt. By default, this parameter is set to none.
Client Cert File	This parameter is applicable only if the target MongoDB server is SSL-enabled and Client PEM File parameter is set to none.In order to collect metrics from the target MongoDB, the eG agent requires client certificate in .p12 format. Hence, specify the full path to the Client certificate file in .p12 format in the Client Cert File text box. For example, the location of this file may be: C:\app\eGurkha\JRE\lib\security\test-client.p12 . To know how to generate .p12 file from Client PEM file, refer to How to import a Certificate that is in the PEM Format? By default, this parameter is set to none.
Client Cert Password	Provide the password for .p12 Client certificate file in the Client Cert Password text box.
AWS Key ID, AWS Secret Key,Confirm Password	If you are monitoring MongoDB server hosted on the AWS cloud, the eG agent has to be configured with the AWS AccessKey ID and Secret Key to connect with the AWS cloud and collect the required metrics. Therefore, Specify the AWS Key ID and AWS Secret Key and confirm the password by re-typing it in the Confirm Password text box. To obtain the AWS Access key and secret key, refer toObtaining AWS Access Key and Obtaining AWS Secret Key.
Atlas URI	MongoDB Atlas is a NoSQL Database-as-a-Service offering in the public cloud. If the target MongoDB server is deployed and managed in MongoDB Atlas, then the eG agent has to be configured with the MongoDB Atlas connection URI,a unique identifier for connecting to a MongoDB server, in the Atlas URI text box to access the target MongoDB server hosted on Atlas and collect the required metrics.

Measurements made by the test
Measurement	Description	Measurement Unit	Interpretation
Flushes	Indicates the number of times the monitored server flushed writes to disk.	Number
Flushes rate	Indicates the rate at which writes were flushed to disk.	Flushes/Sec	A high value is desired, as it indicates that changes are written to disk frequently. Frequent disk writes will minimize the data loss that may occur if the server abnormally exits.
Average flush time	Indicates the average time taken by the server to flush writes to disk.	Seconds	If the value of this measure is over 1000 milliseconds, it is a cause for concern, as it indicates that writing to disk is taking a long time. The most common causes for high flush time are: Disk not having enough IOPS to handle the load; Massive write activities happen all at once; Normally a flush should not take more than 1000 ms, if it does, it is likely the amount of data flushing to disk is too large for the disk to handle. With journaling enabled, which is standard on production MongoDB service, write operations go to journal file on disk and take away valuable disk I/O needed for flushing and page fault. To resolve this issue, you can do one or all of the following: Upgrade to disk with higher IOPS, e.g. SSD / Flash Array. Separate journal file and data file onto separate drives, to free up disk I/O taken by journal file. Spikes in background flush occur when a large amount of dirty pages needed to flush to disk. Tune the application to spot for errors, or disburse writes to a longer time span.
Last flush time	Indicates the time taken by the last disk write.	Seconds	If this value is close to the Average flush time, it could indicate that the slowness occurred during the last flush. Investigating the last flush can provide pointers to why it took more time than normal.