Toward a Formalization of the Process to Select IMIA Yearbook Best Papers

BACKGROUND Each year, the International Medical Informatics Association Yearbook recognizes significant scientific papers, labelled as "best papers", published the previous year in the subfields of biomedical informatics that correspond to the different section topics of the journal. For each section, about fifteen pre-selected "candidate" best papers are externally peer-reviewed to select the actual best papers. Although based on the available literature, little is known about the pre-selection process. OBJECTIVE To move toward an explicit formalization of the candidate best papers selection process to reduce variability in the literature search across sections and over years. METHODS A methodological framework is proposed to build for each section topic specific queries tailored to PubMed and Web of Science citation databases. The two sets of returned papers are merged and reviewed by two independent section editors and citations are tagged as "discarded", "pending", and "kept". A protocolized consolidation step is then jointly conducted to resolve conflicts. A bibliographic software tool, BibReview, was developed to support the whole process. RESULTS The proposed search strategy was fully applied to the Decision Support section of the 2013 edition of the Yearbook. For this section, 1124 references were returned (689 PubMed-specific, 254 WoS-specific, 181 common to both databases) among which the 15 candidate best papers were selected. CONCLUSIONS The search strategy for determining candidate best papers for an IMIA Yearbook's section is now explicitly specified and allows for reproducibility. However, some aspects of the whole process remain reviewer-dependent, mostly because there is no characterization of a "best paper".

[1]  Reinhold Haux,et al.  Individualization, globalization and health - about sustainable information technologies and the aim of medical informatics , 2006, Int. J. Medical Informatics.

[2]  Amit X. Garg,et al.  MEDLINE clinical queries are robust when searching in recent publishing years , 2013, J. Am. Medical Informatics Assoc..

[3]  J. Carlin,et al.  Bias, prevalence and kappa. , 1993, Journal of clinical epidemiology.

[4]  Cord Spreckelsen,et al.  The publication echo: Effects of retrieving literature in PubMed by year of publication , 2010, Int. J. Medical Informatics.

[5]  Jiwon Oh,et al.  Titles versus titles and abstracts for initial screening of articles for systematic reviews , 2013, Clinical epidemiology.

[6]  I. Masic,et al.  A Short Factography About IMIA and EFMI , 2014, Acta informatica medica : AIM : journal of the Society for Medical Informatics of Bosnia & Herzegovina : casopis Drustva za medicinsku informatiku BiH.

[7]  J. Bouaud,et al.  A Medical Informatics Perspective on Clinical Decision Support Systems , 2013, Yearbook of Medical Informatics.

[8]  Dean Giustini,et al.  Google Scholar is not enough to be used alone for systematic reviews , 2013, Online journal of public health informatics.

[9]  Casimir A. Kulikowski,et al.  Case Report: Developing and Evaluating Criteria to Help Reviewers of Biomedical Informatics Manuscripts , 2003, J. Am. Medical Informatics Assoc..

[10]  J. Higgins.  Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration , 2011 .

[11]  M. Samore,et al.  A Sustainable Strategy to Prevent Misuse of Antibiotics for Acute Respiratory Infections , 2012, PloS one.

[12]  R Haux,et al.  Medical Imaging Informatics and Medical Informatics: Opportunities and Constraints , 2002, Methods of Information in Medicine.

[13]  Elske Ammenwerth,et al.  Towards clinical bioinformatics: Advancing genomic medicine with informatics methods and tools - Findings from the IMIA Yearbook of Medical Informatics 2004 , 2004 .

[14]  J. Sim,et al.  The kappa statistic in reliability studies: use, interpretation, and sample size requirements. , 2005, Physical therapy.

[15]  Martin Boeker,et al.  Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough , 2013, BMC Medical Research Methodology.

[16]  R Haux,et al.  Medical Informatics and the Quality of Health: New Approaches to Support Patient Care , 2003, Methods of Information in Medicine.

[17]  R. Hunt,et al.  Systematic Reviews: The Good, the Bad, and the Ugly , 2009, The American Journal of Gastroenterology.

[18]  I. Olkin,et al.  Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement , 1999, The Lancet.

[19]  Marie-Christine Jaulent,et al.  Improving information retrieval using Medical Subject Headings Concepts: a test case on rare and chronic diseases. , 2012, Journal of the Medical Library Association : JMLA.

[20]  Zhiyong Lu,et al.  Evaluation of query expansion using MeSH in PubMed , 2009, Information Retrieval.

[21]  Phil Edwards,et al.  Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records , 2002, Statistics in medicine.

[22]  C A Kulikowski,et al.  Discussion of "Biomedical informatics: we are what we publish". , 2013, Methods of information in medicine.

[23]  Stéfan Jacques Darmoni,et al.  Optimization of the PubMed Automatic Term Mapping , 2009, MIE.

[24]  Lutz Bornmann,et al.  Does the Journal Peer Review Select the “Best” from the Work Submitted? The State of Empirical Research , 2010 .

[25]  S J Darmoni,et al.  CISMeF: A Structured Health Resource Guide , 2000, Methods of Information in Medicine.

[26]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration , 2013 .

[27]  Saoussen Sakji,et al.  Improving information retrieval with multiple health terminologies in a quality-controlled gateway , 2013, Health Inf. Sci. Syst..

[28]  Matthew E Falagas,et al.  Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses , 2007, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[29]  R Haux,et al.  Towards Clinical Bioinformatics: Advancing Genomic Medicine with Informatics Methods and Tools , 2004, Methods of Information in Medicine.

[30]  J. Ménard,et al.  A comparative analysis of four clinical guidelines for hypertension management , 2008, Journal of Human Hypertension.

[31]  Carla E. Brodley,et al.  Semi-automated screening of biomedical citations for systematic reviews , 2010, BMC Bioinformatics.

[32]  J A Kors,et al.  Mapping the Domain of Medical Informatics , 2009, Methods of Information in Medicine.

[33]  A. Garg,et al.  Dialysis search filters for PubMed, Ovid MEDLINE, and Embase databases. , 2012, Clinical journal of the American Society of Nephrology : CJASN.

[34]  D. Cook,et al.  Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions , 1997, Annals of Internal Medicine.

[35]  Zhiyong Lu,et al.  PubMed and beyond: a survey of web tools for searching biomedical literature , 2011, Database J. Biol. Databases Curation.

[36]  C. Kulikowski,et al.  The Challenge of Ubiquitous Computing in Health Care: Technology, Concepts and Solutions , 2005, Methods of Information in Medicine.