Information Extraction: Robust Mention Detection Systems

Information-extraction (IE) research typically focuses on clean-text inputs. However, as we will see in this chapter, an IE engine serving real-world applications yields a high rate of false alarms, due to noisy, less-well-formed input. For example, an application processing output from a multilingual media monitoring system (e.g., TV broadcast) will have to deal with noisy input as well as ­inaccurate automatic transcription and translation. The resulting presence of non-target-language text in this case, and non-language material interspersed in data from other applications, raise the research problem of making IE robust to such noisy input text. This chapter addresses an important IE task: improving robustness to noise for mention detection (MD). We describe the augmentation of an existing statistical MD system to reduce false alarms in the spurious passages while maintaining performance on clean input, and even improving recall. The diverse nature of input noise leads us to pursue a multi-faceted approach to robustness. We describe a multi-stage approach to robustness, reflecting the diverse nature of input noise. Detection-error-trade-off analysis is used to evaluate a MD system. In one experiment, with English as the target language, we find that on inputs from other Latin-alphabet languages, we can eliminate 97–98% of false alarms compared to an English-only baseline system, at various fixed miss rates. In another experiment, modeling situations in which genre-specific training is infeasible, we process real data drawn from a financial-transactions text containing mixed languages and data-set codes. On these data, because annotations for data sets like this are typically not available for training a mention detector, we did not include any portion of this data set in the training of the system, yet still can eliminate 60% of the false alarms at various miss rates, compared to the baseline system. These gains come with virtually no loss in accuracy on clean English text.