DFS Replication Folders Test

DFS Replication is an efficient, multiple-master replication engine that you can use to keep folders synchronized between servers across limited bandwidth network connections. 

DFS Replication uses a compression algorithm known as remote differential compression (RDC). RDC detects changes to the data in a file and enables DFS Replication to replicate only the changed file blocks instead of the entire file.

To use DFS Replication, you must create replication groups and add replicated folders to the groups. Replication groups, replicated folders, and members are illustrated in the following figure.

Figure 1 : How DFS replication works

This figure shows that a replication group is a set of servers, known as members , which participates in the replication of one or more replicated folders. A replicated folder is a folder that stays synchronized on each member. In the figure, there are two replicated folders: Projects and Proposals. As the data changes in each replicated folder, the changes are replicated across connections between the members of the replication group. The connections between all members form the replication topology.

DFS Replication uses staging folders to act as caches for new and changed files to be replicated from sending members to receiving members. The sending member begins staging a file when it receives a request from the receiving member. The process involves reading the file from the replicated folder and building a compressed representation of the file in the staging folder. This is the staged file. After being constructed, the staged file is sent to the receiving member; if remote differential compression [RDC] is used, only a fraction of the staging file might be replicated. The receiving member downloads the data and builds the file in its staging folder. After the file has completed downloading on the receiving member, DFS Replication decompresses the file and installs it into the replicated folder. Each replicated folder has its own staging folder, which by default is located under the local path of the replicated folder in the DfsrPrivate\Staging folder. 

DFS Replication uses a "last-writer wins" method for determining which version of a file to keep when a file is modified on two or more members. The losing file is stored in the Conflict and Deleted folder on the member that resolves the conflict. This member might not be the member where the changes originated. Each replicated folder has its own Conflict and Deleted folder, which is located under the local path of the replicated folder in the DfsrPrivate\ConflictandDeleted folder.

Slow replication can cause replicated folders to remain out-of-sync across members for long time periods. Because of this, users may end up receiving an inconsistent/incomplete view of data. Typically, replication may slow down owing to the following:

  • The lack of adequate bandwidth resources;
  • Staging and conflict and deleted folders that have not been sized to deal with high levels of replication activity;

By closely monitoring these factors, administrators can detect bottlenecks to replication early and take pre-emptive measures. This is where the DFS Replication Folders test helps. This test auto-discovers replicated folders and for each folder reports the bandwith saved during replication and tracks the growth in size of the associated staging folders and config and deleted items folders. This way, administrators can proactively detect probable slowdowns in replication. In addition, they can pinpoint what is causing the slowdown and which replication folders will be affected by this.

Target of the test : A server that hosts the DFS namespace  (this can even be a server that contains the Dfs root or a replica of it)

Agent deploying the test : An internal agent

Outputs of the test : One set of results for each replicated folder

Configurable parameters for the test
Parameters Description

Test period

This indicates how often should the test be executed.

Host

The host for which the test is to be configured.

Port

The port at which the specified host listens to.

Measurements made by the test
Measurement Description Measurement Unit Interpretation

Bandwidth savings using DFS replication

Indicates the percentage of bandwidth that was saved by the DFS Replication service for the replicated folder.

.

Percent

A high degree of compression results in high bandwidth savings and faster/inexpensive replication. A high value is therefore desired for this measure. A low value is often an outcome of a poor compression algorithm.

Conflict files generated

Indicates the number of files and folders in this replicated folder that were moved to the conflict and deleted folder during the last measurement period.

Number

The value of this measure is a good indicator of how often the files in a replication folder are changed.

Conflict files cleanups

Indicates the number of conflict loser files and folders that were deleted from the conflict and deleted folder of this replicated folder during the last measurement period.

Number

The Conflict and Deleted folder has high and low watermarks (90 percent and 60 percent of Conflict and Deleted folder quota, respectively) that govern cleanup and excessive usage of the folder.

This implies that if the size of the Conflict and Deleted folder is over 90% of the quota configured for it, DFS will start deleting files from the folder automatically and will keep at it until the size of the folder touches its low watermark – i.e., until size becomes 60% of the configured quota.

If the value of this measure is consistently high for a replicated folder, it can only mean that the Conflict and Deleted folder is continuously growing in size, thus forcing DFS to cleanup old files from it on a regular basis.

This could also indicate that files in the corresponding replicated folder are changing very often, thus adding to the contents and the size of the Conflict and Deleted folder. Another reason for a large number of cleanups could be a low quota configured for the Conflict and Deleted folder.

Deleted files generated

Indicates the number of replicated deleted files and folders that were moved from this replicated folder to the conflict and deleted folder during the last measurement period.

Number

The Conflict and Deleted folder can also be used to store files that are deleted from replicated folders.

 

Deleted files cleanups

Indicates the number of replicated deleted files and folders that were deleted from the conflict and deleted folder of this replicated folder during the last measurement period.

.

Number

Deleted files are treated like conflict files in that they are purged when the Conflict and Deleted folder reaches 90 percent of the configured quota.

If the size of the Conflict and Deleted folder grows above 90% of its configured quota, DFS will start deleting files from the folder automatically and will keep at it until the size of the folder touches its low watermark – i.e., until size becomes 60% of the configured quota. 

If the value of this measure is consistently high for a replicated folder, it can only mean that the Conflict and Deleted folder is continuously growing in size, thus forcing DFS to cleanup old files from it on a regular basis.

This could also indicate that files in the corresponding replicated folder are either changing or are being deleted very often, thus adding to the contents and the size of the Conflict and Deleted folder.

Staging files generated

Indicates the number of times files in this replicated folder were staged during the last measurement period.

Number

The value of this measure is a good indicator of the number of times files were added to or changed in a replication folder.

A high value is therefore indicative of a high level of replication activity. A consistent and significant increase in the value of this measure over time could also indicate that staging files are not being sent to the receiving member as quickly as they are created by the sending member – this in turn could indicate a bottleneck in replication.

Staging files cleanups

Indicates the number of files and folders that were cleaned up from the staging folder of this replication folder by the DFS Replication service during the last measurement period.

Number

The DFS  Replication service stages files and folders in the staging folder before they are replicated, and automatically cleans up the staging folder when it exceeds  a pre-configured threshold of the quota.

If the value of this measure remains high consistently, it could indicate that the staging folder is rapidly exhausting its configured quota size, causing DFS to keep deleting old staging files from the folder. While this can happen if the level of replication activity is high, it can also happen if the staging folder is configured with a low quota size.  If a staging folder quota is configured to be too small, DFS Replication might consume additional CPU and disk resources to regenerate the staged files. Replication might also slow down because the lack of staging space can effectively limit the number of concurrent transfers with partners.

Conflict space in use

Indicates  the total size of conflict loser files and folders in the conflict and deleted folder of this replicated folder during the last measurement period.

MB

The quota size of the Conflict and Deleted folder is 660 MB by default; however, this can be changed. Also, the Conflict and Deleted folder is configured with a high watermark of 90% and a low watermark of 60% of the quota. If the value of this measure is over 90% of the default/configured (as the case may be) quota size, then DFS will start purging files from the conflict and deleted folder until its size falls to 60% of the quota.

Compare the value of this measure across replication folders to know which replication folder’s conflict and deleted folder is growing close to its configured quota size.

Deleted space in use

Indicates the total size of replicated deleted files and folders currently in the conflict and deleted folder of this replicated folder.

MB

Deleted files are treated like conflict files in that they are purged when the Conflict and Deleted folder reaches 90 percent of the configured quota.

Staging space in use

Indicates the total size of the files and folders in the staging folder of this replication folder during the last measurement period.

MB

The default size of each staging folder is 4,096 MB. This is not a hard limit, however. It is only a quota that is used to govern cleanup and excessive usage based on high and low watermarks (90 percent and 60 percent of staging folder size, respectively). This quota can be changed. 

If the value of this measure reaches 90% of the configured quota, the oldest staged files are purged until the staging folder reaches 60 percent of the configured quota.

For good operational performance, increasing the quota size of a staging folder is recommended when you have multiple large files that change frequently. Microsoft also recommends that you increase the staging folder quota on hub members that have many replication partners. 

If a staging folder quota is configured to be too small, DFS Replication might consume additional CPU and disk resources to regenerate the staged files. Replication might also slow down because the lack of staging space can effectively limit the number of concurrent transfers with partners.