An Evaluation of Text Matching Systems for Text Excerpts of Varying Scope

When large text collections must be processed, it is not possible to limit the scope of the subject matter of interest. In such a situation the standard content analysis methods that are based on the use of knowledge bases to represent the relevant subject areas are not applicable. Necessarily, the text themselves must then serve as the main basis for the content analysis operations. Experiments are described in this note designed to evaluate text matching operations for text excerpts of varying scope, including in particular text paragraphs and text sentences extracted from book size materials. The evaluation shows that when the global text similarity between distinct text paragraphs is high, while at the same time local similarities also exist for particular text sentences included in these paragraphs, the presumption is that the paragraphs cover related subject matter. One concludes that text matching systems may prove useful for text linking and information retrieval.