Document Processors Test

The document processor takes the content that the crawler has collected and sent via the content distributor and organizes it into searchable text by sending the content through the indexing pipeline. As such, the document processor is the mechanism that pushes items through the pipeline in which different processors perform different, specific tasks on the content. Document processors can perform many tasks, such as extracting text from files, breaking the text into words, identifying languages, and extracting entities. By the end of the indexing pipeline, several unique processors are passed and the document is in a FiXML format that can be stored in the index and made searchable. The Document Processors send the converted document to the Indexing Dispatchers which in turn sends it to the Indexer that generates a searchable Index.

This test reports the current status of each of the document processors configured on the FAST Search 2010 server for SharePoint. This way, you can quickly figure out which document processor is doing what. Additionally, this test reports the number of times the Document Processor is initialized and the number of times the reassignment of the content distributor dispatcher has been completed/started.

Target of the test : A FAST Search Server 2010 for SharePoint

Agent deploying the test : An internal agent

Outputs of the test : One set of results for each document processor on the FAST Search Server 2010 for SharePoint farm.

Configurable parameters for the test
Parameter Description

Test period

How often should the test be executed

Host

The host for which the test is to be configured.

Port

Refers to the port used by the specified host. By default, this is 13280.
Measurements made by the test
Measurement Description Measurement Unit Interpretation

Indexing dispatcher input queue state

Indicates the current state of the Indexing Dispatcher input queue i.e., the queue through which the converted document is sent to the Indexing Dispatcher by the Document Processor.

 

This measure reports the value Full if there are one/more documents in the Indexing dispatcher input queue and Not Full if no documents are there in the queue.

The values reported by this measure and their numeric equivalents are available in the table below:

Measure Value Numeric Value
Full 1
Not Full 0

Note:

This measure reports the Measure Value listed in the table above to indicate the state of the Indexing dispatcher input queue. However, in the graph of this measure, the queue state is indicated using only the Numeric Values listed in the above table.

Is initializing?

Indicates whether/not this document processor currently initializing.

Number

The values reported by this measure and their numeric equivalents are available in the table below:

Measure Value Numeric Value
Yes 1
No 0

Note:

This measure reports the Measure Values listed in the table above to indicate whether/not the Document Processor is initializing. However, in the graph of this measure, the initializing state of the Document Processor is indicated using only the Numeric Values listed in the above table.

Is processing a set of operations?

Indicates whether/not this Document Processor is currently processing a set of operations.

 

The values reported by this measure and their numeric equivalents are available in the table below:

Measure Value Numeric Value
Yes 1
No 0

Note:

This measure reports the Measure Values listed in the table above to indicate whether/not the Document Processor is processing a set of operations. However, in the graph of this measure, the same is indicated using only the Numeric Values listed in the above table.

Is submitting a set of operations to the indexing dispatcher?

Indicates whether/not the set of operations were submitted to the Index dispatcher by this Document Processor.

 

The values reported by this measure and their numeric equivalents are available in the table below:

Measure Value Numeric Value
Yes 1
No 0

Note:

This measure reports the Measure Values listed in the table above to indicate whether/not the Document Processor is submitting a set of operations to the index dispatcher. However, in the graph of this measure, the same is indicated using only the Numeric Values listed in the above table.

Is sharePoint pipeline initialized?

Indicates whether/not the sharepoint pipeline is initialized.

 

The values reported by this measure and their numeric equivalents are available in the table below:

Measure Value Numeric Value
Yes 1
No 0

Note:

This measure reports the Measure Values listed in the table above to indicate whether/not the sharepoint pipeline is initialized. However, in the graph of this measure, the same is indicated using only the Numeric Values listed in the above table.

Total number of initializations

Indicates the number of times this document processor was initialized.

Number

 

Reassignments of the contentdistributor dispatcher has been completed

Indicates the number of times the reassignment of the content distributor dispatcher is completed.

Number

 

Nameserver resolve of contentdistributor dispatcher scope

Indicates the total number of invocations of the nameserver resolve of the contentdistributor dispatcher.

Number

 

Nameserver resolve of contentdistributor master scope

Indicates the total number of invocations of the nameserver resolve of the primary contentdistributor.

Number