Corpus-based Evaluation of a French Spelling and Grammar Checker

This article describes an evaluation method for spelling and grammar checkers and gives the results of its application to two French checkers. The evaluation process follows closely the ISO/IEC and EAGLES guidelines, and defines precisely the evaluation metrics, so that they can be easily reproduced. The choice of professional translators as user profile entails the use of a corpus of spelling mistakes, which was collected and annotated. The metrics are divided into three sets: classification of perfect vs. imperfect sentences; detection of mistakes; correction of mistakes. The results show in which respect the two systems are the most adapted to the user needs, and the points on which they could be improved.