Ogmios: a scalable NLP platform for annotating large web document collections

While NLP tools are now widely available, their use can be problematic considering the lack of homogeneity of their input/output format, the variation of the granularity of their information, but also the difficulties to process large amounts of heterogeneous documents in a reasonable time, and their tunability to a specific domain. To address these problems, we propose a configurable platform to enrich very large collections of French and English specialised documents. The platform is a modularized framework. Each module carries out an annotation step by using existing NLP tools and can be tuned to a domain by adding specific resources: named entity recognition, sentence and word segmentation, lemmatisation, POS tagging, term tagging and parsing. Linguistic annotations are recorded in a stand-off XML format. We focus on the robustness of the annotation process to help the creation of annotated corpora from the web. We have tested the scalability of the platform on two collections of 55,329 biomedical web documents (107 millions of words) and 47,393 Search Engine News (13 millions of words). The collections have been annotated until the term tagging, respectively in 35 hours and 3 hours.