Evaluation of Intelligent Information Retrieval Tools for Unstructured Police Case Reports

Abstract—The Belgian Federal Police (BFP) already using anarray of advanced and task-specific tools for collection, dispersaland exploitation of highly structured information, is now shiftingattention towards intelligent information retrieval technologiesfor exploiting vast amounts of unstructured information sources(police case reports), from which further knowledge could beobtained for analysis purposes. The INFO - NS research project, isaimed to provide an objective study of the applicability of miningand decision support tools for the BFP. More specifically, it isstudied whether information retrieval, extraction and classifica-tion tools might leverage intelligence and decision support whileexploiting the information that is contained in vast amounts offree text material and linking it with any coexisting, structureddata sets that are currently in use. This article provides an insightinto the requirements of the BFP and police in general withregard to a like decision support system, gives an overview ofour evaluation approach and presents one aspect of evaluationcriteria aimed at testing the functional conformity of such toolsagainst a set of functional requirements.Index Terms—information retrieval, text mining,law enforce-ment, Knowledge representation, evaluation techniques, multi-criteria evaluation