Indexer Partition Test

The overall index created by an Indexer has two levels of partitioning: index columns and index partitions.

When the complete index is too large to reside on one server, you can split it into multiple disjoint index columns. Within each index column the indexer uses a partitioning of the index in order to handle large number of indexed items with low indexing and query latency. Index partitioning enables administrators to spread the load for queries across multiple servers. This is achieved by creating subsets of an index, and propagating individual subsets to different index servers. This partitioning is dynamic and handled internally on each index server. When the query matching component evaluates a query, each partition runs in a separate thread. The default number of partitions is 5. In order to handle more than 15 million items per server (column), you must configure a larger number of partitions.

To check whether sufficient indexer partitions are up and running to serve the query load effectively, you need to do the following:

• Run periodic checks on the state of the individual partitions to identify the unavailable partitions;
• Continuously monitor the load on the partitions and how quickly each partition processes the load, so that probable overload conditions and processing bottlenecks  can be identified and fixed early.

The Indexer Partition test helps achieve all of the above.

Target of the test : A FAST Search Server 2010 for SharePoint

Agent deploying the test : An internal agent

Outputs of the test : One set of results for every indexer partition configured on each query matching server in the FAST Search Server 2010 for SharePoint farm.

Configurable parameters for the test

Parameter	Description
Test period	How often should the test be executed
Host	The host for which the test is to be configured.
Port	Refers to the port used by the specified host. By default, this is 13280.

Measurements made by the test

Measurement	Description	Measurement Unit	Interpretation
Number of active items	Indicates the number of active items on this index partition.	Number	The Index partitions 0 and 1 should have less than 1 million items each, preferably even less in order to keep indexing latency low. In periods with high item throughput, indexing latency will be reduced and these partitions will be larger, as this is more optimal for overall throughput. Items will although automatically be rearranged into the higher numbered partitions during periods with lighter load.
Number of items indexed	Indicates the number of items that are indexed per second for this partition.	Number	A consistent decrease in the value of this measure could indicate a processing bottleneck.
Pending exclusion	Indicates the number of items that are to be excluded from indexing in this index partition.	Number	This translates to items that are in the index but that should not be searchable (deleted, present in other partitions, etc).
Current state of partition	Indicates the current state of this partition.	Number
Total number of items	Indicates the total number of items in this partition including the items on the excluded lists.	Number	This measure is a good indicator of the total load on a partition. Compare the value of this measure across partitions to isolate the overloaded partitions. If one/more partitions are found to be handling abnormally high load than the rest, it could indicate that load is unevenly distributed across the partitions. You may hence have to configure more partitions to handle the load.