Generating a dynamic hypertext environment with n-gram analysis

Methods and tools for finding documents relevant to a user’s needs in a very large corpus of documents can be found in the information retrieval, library science, and hypertext communities. Typically, these systems provide retrieval capabilities for fairly static corpora, their algorithms are language dependent, and they don’t perform well when presented with misspelled words or text that has been degraded by optical character recognition techniques. In addition, the capabilities needed by a variety of researchers, analysts, and investigative reporters from these methods and tools go beyond the traditional indexing systems. The ability to find related documents, to query using natural language, and to filter out desired information suggest a more interactive and robust user interface. We describe our work in progress for generating a dynamic hypertext environment that provides the capabilities of relating, querying, and iiltering in a largescale, dynamic environment that is robust in the sense that it is language independent and capable of dealing with misspelled words.