Selective text utilization and text traversal

Many large collections of full-text documents are currently stored in machine-readable form and processed automatically in various ways. These collections may include different types of documents, such as messages, research articles, and books, and the subject matter may vary widely. To process such collections, robust text analysis methods must be used, capable of handling materials in arbitrary subject areas, and flexible access must be provided to texts and text excerpts of varying size. In this study, global text comparison methods are used to identify similarities between text elements, followed by local context-checking operations that resolve ambiguities and distinguish superficially similar texts from texts that actually cover identical topics. A linked text structure is then created that relates similar texts at various levels of detail. In particular, text links are available for full texts, as well as text sections, paragraphs, and sentence groups. The linked structures are usable to identify important text passages, to traverse texts selectively both within particular documents and between documents, and to provide flexible text access to large text collections in response to various kinds of user needs. An automated 29-volume encyclopedia is used as an example to illustrate the text accessing and traversal operations.

[1]  Hans Peter Luhn,et al.  The Automatic Creation of Literature Abstracts , 1958, IBM J. Res. Dev..

[2]  Phyllis B. Baxendale,et al.  Machine-Made Index for Technical Literature - An Experiment , 1958, IBM J. Res. Dev..

[3]  H. P. Edmundson,et al.  Problems in automatic abstracting , 1964, CACM.

[4]  Lois L. Earl,et al.  Experiments in automatic extracting and indexing , 1970, Inf. Storage Retr..

[5]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[6]  James E. Rush,et al.  Automatic abstracting and indexing. II. Production of indicative abstracts by application of contextual inference and syntactic coherence criteria , 1971 .

[7]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[8]  John O'Connor,et al.  Answer-passage retrieval by text searching , 1980, J. Am. Soc. Inf. Sci..

[9]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[10]  Udo Hahn,et al.  Text condensation as knowledge base abstraction , 1988, [1988] Proceedings. The Fourth Conference on Artificial Intelligence Applications.

[11]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[12]  Gerard Salton,et al.  Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer , 1989 .

[13]  Jay David Bolter,et al.  Writing Space: the Computer, Hypertext, and the History of Writing , 1990 .

[14]  Chris D. Paice,et al.  Constructing literature abstracts by computer: Techniques and prospects , 1990, Inf. Process. Manag..

[15]  Mark Bernstein.  An Apprentice That Discovers Hypertext Links , 1990, ECHT.

[16]  P. Delany,et al.  Hypermedia and literary studies , 1991 .

[17]  G Salton,et al.  Global Text Matching for Information Retrieval , 1991, Science.

[18]  G Salton,et al.  Developments in Automatic Text Retrieval , 1991, Science.

[19]  Jay David Bolter,et al.  Architectures for volatile hypertext , 1991, HYPERTEXT '91.

[20]  Gerard Salton,et al.  Automatic text structuring and retrieval-experiments in automatic encyclopedia searching , 1991, SIGIR '91.

[21]  Ben Shneiderman,et al.  Structural analysis of hypertexts: identifying hierarchies and useful metrics , 1992, TOIS.

[22]  James Allan,et al.  Approaches to passage retrieval in full text information systems , 1993, SIGIR.

[23]  Marilyn Deegan,et al.  Writing space: The computer, hypertext, and the history of writing , 1993 .

[24]  Alan F. Smeaton,et al.  Information retrieval from hypertext using dynamically planned guided tours , 1993, ECHT '92.