Texts vary not only by topic, but by style; indeed, often the variation between texts 'about the same thing' can be just as noticeable as the variation between texts 'about different things'. Some facets of this variation are quite easy to detect, and quite predictable when applied to categorization of texts by genre, functional style, or-tentatively-quality. Making use of such variation in a retrieval context is quite straightforward in principle; our work consists of an implementation of a visualization tool for document databases. The issues addressed include: (1) choice of stylistic items to investigate; (2) composition of dimensions of variation; and (3) judicious naming of dimensions for presentation. We use use principal components analysis to combine our quite large number of stylistic items into two most significant dimensions of variation and plot the document space under consideration into a plane. This space can be used as a first or last filter in an information retrieval task. The composition of the most significant dimensions is naturally corpus dependent, as is the naming of them: our work is tested on Internet and TREC data.
[1]
Jussi Karlgren,et al.
Stylistic Variation in an Information Retrieval Experiment
,
1996,
ArXiv.
[2]
George R. Klare,et al.
The measurement of readability
,
1963
.
[3]
D. Biber.
A typology of English texts
,
1989
.
[4]
Robert M. Losee,et al.
Text Windows and Phrases Differing by Discipline, Location in Document, and Syntactic Structure
,
1996,
Inf. Process. Manag..
[5]
Douglas Biber,et al.
Variation across speech and writing: Methodology
,
1988
.
[6]
Jussi Karlgren,et al.
Recognizing Text Genres With Simple Metrics Using Discriminant Analysis
,
1994,
COLING.