论文信息 - An Experiment in Detection and Correction of Malapropisms Through the Web

An Experiment in Detection and Correction of Malapropisms Through the Web

Malapropism is a type of semantic errors. It replaces one content word by another content word similar in sound but semantically incompatible with the context and thus destructing text cohesion. We propose to signal a malapropism when a pair of syntactically linked content words in a text exhibits the value of a specially defined Semantic Compatibility Index (SCI) lower than a predetermined threshold. SCI is computed through the web statistics of occurrences of words got together and apart. A malapropism detected, all possible candidates for correction of both words are taken from precompiled dictionaries of paronyms, i.e. words distant a letter or a few prefixes or suffixes from one another. Heuristic rules are proposed to retain only a few highly SCI-ranked candidates for the user's decision. The experiment on mala-propism detection and correction is done for a hundred Russian text fragments—mainly from the web newswire—in both correct and falsified form, as well as for several hundreds of correction candidates. The raw statistics of occurrences is taken from the web searcher Yandex. Within certain limitations, the experiment gave very promising results.

Igor A. Bolshakov

[1] Alexander F. Gelbukh,et al. Compilation of a Spanish Representative Corpus , 2002, CICLing.

[2] Hinrich Schütze,et al. Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[3] Adam Kilgarriff,et al. Introduction to the Special Issue on the Web as Corpus , 2003, CL.

[4] Janusz Kacprzyk,et al. Advances in Web Intelligence , 2003, Lecture Notes in Computer Science.

[5] Alexander F. Gelbukh,et al. On Detection of Malapropisms by Multistage Collocation Testing , 2003, NLDB.

[6] Graeme Hirst,et al. Correcting real-word spelling errors by restoring lexical cohesion , 2005, Natural Language Engineering.

[7] Alexander F. Gelbukh,et al. On Correction of Semantic Errors in Natural Language Texts with a Dictionary of Literal Paronyms , 2004, AWIC.

[8] Ted Pedersen,et al. A Decision Tree of Bigrams is an Accurate Predictor of Word Sense , 2001, NAACL.

[9] Frank Keller,et al. Using the Web to Obtain Frequencies for Unseen Bigrams , 2003, CL.

[10] Igor A. Bolshakov,et al. Getting One's First Million ...Collocations , 2004, CICLing.

[11] Philip W. Goetz. The New Encyclopaedia Britannica , 1991 .

[12] Igor Mel’čuk,et al. Dependency Syntax: Theory and Practice , 1987 .

[13] Graeme Hirst,et al. Lexical chains as representations of context for the detection and correction of malapropisms , 1995 .