Assessing the performance of Olelo, a real-time biomedical question answering application

Question answering (QA) can support physicians and biomedical researchers to find answers to their questions in the scientific literature. Such systems process large collections of documents in real time and include many natural language processing (NLP) procedures. We recently developed Olelo, a QA system for biomedicine which includes various NLP components, such as question processing, document and passage retrieval, answer processing and multi-document summarization. In this work, we present an evaluation of our system on the the fifth BioASQ challenge. We participated with the current state of the application and with an extension based on semantic role labeling that we are currently investigating. In addition to the BioASQ evaluation, we compared our system to other on-line biomedical QA systems in terms of the response time and the quality of the answers.

[1]  Hwee Tou Ng,et al.  Domain adaptation for semantic role labeling in the biomedical domain , 2010, Bioinform..

[2]  Georgios Balikas,et al.  An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition , 2015, BMC Bioinformatics.

[3]  Mariana L. Neves.  HPI in-memory-based Database System in Task 2b of BioASQ , 2014, CLEF.

[4]  Ulf Leser,et al.  Question answering for biology. , 2015, Methods.

[5]  Mariana L. Neves.  HPI Question Answering System in the BioASQ 2015 Challenge , 2015, CLEF.

[6]  Hong Yu,et al.  AskHERMES: An online question answering system for complex clinical questions , 2011, J. Biomed. Informatics.

[7]  Michael A Bauer,et al.  Usability survey of biomedical question answering systems , 2012, Human Genomics.

[8]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[9]  Mariana L. Neves,et al.  Olelo: a web application for intuitive exploration of biomedical literature , 2017, Nucleic Acids Res..

[10]  Mariana L. Neves,et al.  Neural Question Answering at BioASQ 5B , 2017, BioNLP.

[11]  T. Rindflesch,et al.  What about trust in the Question Answering world ? , 2009 .

[12]  Mariana L. Neves,et al.  HPI Question Answering System in BioASQ 2016 , 2016 .

[14]  Patrick Ruch,et al.  Deep Question Answering for protein annotation , 2015, Database J. Biol. Databases Curation.

[15]  Mariana L. Neves,et al.  Entity-Supported Summarization of Biomedical Abstracts , 2016, BioTxtM@COLING 2016.

[16]  Hyoil Han,et al.  Biomedical question answering: A survey , 2010, Comput. Methods Programs Biomed..