Error-tolerant Finite State Recognition

Error-tolerant recognition enables the recognition of strings that deviate slightly fro� any string in the regular set recognized by the underlying finite state recognizer. In the context of natural language processing, it has applications in error-tolerant morphological analysis, and spe�g correction. After a description of the concepts and algorithms involved, we give examples from these two applications: In morphological analysis, error-tolerant recognition allows misspelled input word forms to � corrected, and morphologically analyzed concurrently. The algorithm can be applied to the moiphological analysis of any language whose morphology is fully captured by a single (and possibly very large) finite state transducer, regardless of the word formation processes (such as agglutination or productive compounding) and morphographemic phenomena involved. We present an .application to error­ tolerant analysis of agglutinative morphology of Turkish words. In spelling correction, error-tolerant recognition can be used to enumerate correct candidate forms from a given misspelled string within a certain edit distance. It can be applied to any language whose morphology is fully described by·a finite state transducer, or with a word list comprising all inflected forms with very large word lists of root· and inflected forms (some containing well over 200,000 forms), generating all candida� solutions within 10 to 45 milliseconds (with edit distance 1 ) on a SparcStation 10/41 . For spelling correction in Turkish, error-tolerant recognition operating with a (circular) recognizerofTurkish words (with about 29,000 states and 1 19,000 transitions) can generate all candidate words in less than 20 milliseconds (with edit distance 1 ). Spelling correction using a recognizer constructed from a large word German list that simulates compounding, also indicates that the approach is applicable in such cases.