Review of the Delphi method in library and information science research

This article presents an introduction to the Delphi method and review of Delphi studies published in the literature of library and information science (LIS).,A review of Delphi studies published between the years of 1971 and 2019 is performed, using studies retrieved from the Library and Information Science Source database. A total of 122 articles were retrieved and evaluated based on the population studied, means of identifying experts, number of participants for each study round, type of Delphi, and type of findings.,General librarians (any type), academic librarians, and information science researchers are the most common populations in LIS Delphi studies. On average (middle 50 percent of studies), 14–36 experts are used in the first round of LIS Delphi studies (median n = 23). Employment in a specific role and publications in scholarly journals are the most common means of identifying experts. Variants of the e-Delphi (online survey/email) method are increasingly common, particularly in LIS Delphi studies that focus on general information science, rather than library, topics. Though LIS Delphi studies are relatively few in number, they have a consistent record of being published in some of the most prestigious LIS journals.,This paper provides an introduction to the Delphi method for LIS research and presents an overview of existing literature in LIS that utilizes the research method. No overview of this extent exists in the LIS literature, and, thus, this paper may serve as an important information source about the method for LIS researchers.

[1]  Lili Luo,et al.  Professional Preparation for “Text a Librarian” , 2012 .

[2]  Olga De Troyer,et al.  A Delphi-based expert judgment method applied to the validation of a mature Agile framework for Web development projects , 2019, Inf. Technol. Manag..

[3]  Rik Crutzen,et al.  An Exploration of Factors Related to Dissemination of and Exposure to Internet-Delivered Behavior Change Interventions Aimed at Adults: A Delphi Study Approach , 2008, Journal of medical Internet research.

[4]  K. Tuamsuk,et al.  Core competencies for information professionals of Thai academic libraries in the next decade (A.D. 2010–2019) , 2011 .

[5]  Hossein Riazi,et al.  Task-specific usability requirements of electronic medical records systems: Lessons learned from a national survey of end-users , 2018, Informatics for health & social care.

[6]  F. Woudenberg An Evaluation of Delphi , 1991 .

[7]  Adam Wright,et al.  Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems , 2011, J. Am. Medical Informatics Assoc..

[8]  Roya Pournaghi,et al.  The factors and criteria for prioritization of GIS utilization by libraries , 2015, Electron. Libr..

[9]  Laura Saunders,et al.  The Future of Information Literacy in Academic Libraries: A Delphi Study , 2009 .

[10]  P. R. Goswami The Application of Leadership and Management Principles and Strategies to a Library and Information Service , 2002 .

[11]  Jay R. Avella,et al.  Delphi Panels: Research Design, Procedures, Advantages, and Challenges , 2016 .

[12]  Tove Faber Frandsen,et al.  Citing the innovative work of the original inventors: An analysis of citations to prior clinical trials , 2017 .

[13]  Kenneth E. Vance Future of Library Education: 1975 Delphi Study. , 1977 .

[14]  Heidi Julien,et al.  Trends in information behavior research, 1999–2008: A content analysis , 2011 .

[15]  Luiz Antonio Joia,et al.  Motivations for the IT Professional Turnaway Intention: A Delphi Approach , 2019, Inf. Syst. Manag..

[16]  Shifra Baruchson-Arbib,et al.  A view to the future of the library and information science profession: A Delphi study , 2002, J. Assoc. Inf. Sci. Technol..

[17]  N W Matheson Perspectives on academic health sciences libraries in the 1980s: indicators from a Delphi study. , 1982, Bulletin of the Medical Library Association.

[18]  K. Howard Emergence of a new method: The Grounded Delphi method , 2018, Library and Information Research.

[19]  M Dwyer,et al.  A Delphi survey of research priorities and identified areas for collaborative research in health sector library and information services UK. , 1999, Health libraries review.

[20]  Kieran Walsh,et al.  Consensus on Quality Indicators of Postgraduate Medical E-Learning: Delphi Study , 2018, JMIR medical education.

[21]  N. Dalkey,et al.  An Experimental Application of the Delphi Method to the Use of Experts , 1963 .

[22]  Samuel DiGangi,et al.  Assessing unidimensionality: A comparison of Rasch modeling, Parallel analysis, and TETRAD , 2007 .

[23]  Zuraidah Abd Manaf Establishing the national digital cultural heritage repository in Malaysia , 2008 .

[24]  David Baker,et al.  Digital library futures: a UK HE and FE perspective , 2006 .

[25]  Martin Jung,et al.  How to improve the delivery of medication alerts within computerized physician order entry systems: an international Delphi study , 2011, J. Am. Medical Informatics Assoc..

[26]  Diljit Singh,et al.  Developing an academic library website model, a designer's checklist, and an evaluative instrument: A Delphi method approach , 2015, Electron. Libr..

[27]  Alice Keller,et al.  Future development of electronic journals: a Delphi survey , 2001, Electron. Libr..

[28]  Christopher G. Reddick,et al.  Understanding electronic government research and smart city: A framework and empirical evidence , 2016, Inf. Polity.

[29]  Amany M. Elsayed Web content strategy in higher education institutions , 2017 .

[30]  Darlene E. Weingand Wisconsin Continuing Education Profile: A Delphi Projection of Needs 1986. , 1989 .

[31]  Noa Aharony,et al.  Views and Dreams: A Delphi Investigation into Library 2.0 Applications , 2009 .

[32]  Robert West,et al.  Identification of Behavior Change Techniques and Engagement Strategies to Design a Smartphone App to Reduce Alcohol Consumption Using a Formal Consensus Method , 2015, JMIR mHealth and uHealth.

[33]  AddisonTom E-commerce project development risks , 2003 .

[34]  James H. Wittebols Empowering Students to Make Sense of an Information Saturated World: The Evolution of Information Searching and Analysis , 2016 .

[35]  Fernando Martín-Alcázar,et al.  Assessing social capital in academic research teams: a measurement instrument proposal , 2019, Scientometrics.

[36]  Lyn Robinson,et al.  Slow Delphi: An investigation into information behaviour and the Slow Movement , 2014, J. Inf. Sci..

[37]  Karla K. Krueger,et al.  The Status of Statewide Subscription Databases , 2012 .

[38]  Yunfei Du Librarians' responses to reading at risk : A Delphi study , 2009 .

[39]  Mayank Yuvaraj,et al.  Ascertaining the factors that influence the acceptance and purposeful use of cloud computing in medical libraries in India , 2016 .

[40]  Mehdi Khosrow-Pour,et al.  Critical issues of Web-enabled technologies in modern organizations , 2001, Electron. Libr..

[41]  T. Tomasik Reliability and validity of the Delphi method in guideline development for family physicians. , 2010, Quality in primary care.

[42]  Maria Pinto,et al.  Information literacy life cycle and its standards and models: A view from Ibero-America , 2017, J. Libr. Inf. Sci..

[43]  Delia Neuman High school students' use of databases: results of a national Delphi study , 1995 .

[44]  Jon Iden,et al.  Setting the Stage for a Successful ITIL Adoption: A Delphi Study of IT Experts in the Norwegian Armed Forces , 2010, Inf. Syst. Manag..

[45]  Chaim Zins Classification schemes of Information Science: Twenty-eight scholars map the field , 2007 .

[46]  H. de Vries,et al.  Identifying Factors for Optimal Development of Health-Related Websites: A Delphi Study Among Experts and Potential Future Users , 2012, Journal of medical Internet research.

[47]  K. D. Joshi,et al.  Knowledge manipulation activities: results of a Delphi study , 2002, Inf. Manag..

[48]  A V Milholland,et al.  Medical assessment by a Delphi group opinion technic. , 1973, The New England journal of medicine.

[49]  J. A. Hill Are Salary Surveys Useful , 1963 .

[50]  Akkanad M. Isaac,et al.  Curricula in information science: Analysis and development , 1971 .

[51]  W. Jessiman,et al.  Prioritising research and dissemination: a Delphi study of NHS Highland midwives , 2012 .

[52]  Murat Koyuncu,et al.  Understanding key skills for information security managers , 2018, Int. J. Inf. Manag..

[53]  Lyn Robinson,et al.  ‘Being in a knowledge space’: Information behaviour of cult media fan communities , 2017, J. Inf. Sci..

[54]  Karina Lovell,et al.  How expert are the experts? An exploration of the concept of 'expert' within Delphi panel techniques. , 2006, Nurse researcher.

[55]  Amy R. Hofer,et al.  Identifying Threshold Concepts for Information Literacy: A Delphi Study. , 2016 .

[56]  Błażej Feret,et al.  THE FUTURE OF THE ACADEMIC LIBRARY AND THE ACADEMIC LIBRARIAN: A DELPHI STUDY RELOADED , 2005 .

[57]  Athena Salaba,et al.  What Is Next for Functional Requirements for Bibliographic Records? A Delphi Study1 , 2009, The Library Quarterly.

[58]  Brian Cape Gathering opinion and initiating debate: the success of the Delphi Method in purely qualitative research , 2009 .

[59]  Justin Dillon,et al.  Exploring Research Themes in Public Engagement Within a Natural History Museum: A Modified Delphi Approach , 2013 .

[60]  Petros A Kostagiolas,et al.  An analytic hierarchy approach for intellectual capital , 2010 .

[61]  J. Fernández-Molina,et al.  An Open Access Policy for the Scientific Output of Cuba’s National Health System , 2012 .

[62]  Amy J. Connolly,et al.  BILI: Building Information Literacy in Ireland , 2013 .

[63]  Ricardo Colomo Palacios,et al.  A Qualitative Study of Hard Decision Making in Managing Global Software Development Teams , 2010, Inf. Syst. Manag..

[64]  F. Hasson,et al.  Enhancing rigour in the Delphi technique research , 2011 .

[65]  Yared Mammo Cherinet Blended skills and future roles of librarians , 2018 .

[66]  Bryan R. Cole,et al.  The Importance of the Stakeholder in Performance Measurement: Critical Processes and Performance Measures for Assessing and Improving Academic Library Services and Programs , 2005 .

[67]  Kalle Lyytinen,et al.  Identifying Software Project Risks: An International Delphi Study , 2001, J. Manag. Inf. Syst..

[68]  Luís Ferreira Pires,et al.  Business Intelligence and Service-oriented Architecture: A Delphi Study , 2010, Inf. Syst. Manag..

[69]  Julie McLeod,et al.  The challenges facing public libraries in the Big Society: The role of volunteers, and the issues that surround their use in England , 2015, J. Libr. Inf. Sci..

[70]  Oded Nov,et al.  Stay on the Wikipedia task: When task-related disagreements slip into personal and procedural conflicts , 2013, J. Assoc. Inf. Sci. Technol..

[71]  Daniela Haluza,et al.  Information and communication technology and the future of healthcare: Results of a multi-scenario Delphi survey , 2019, Health Informatics J..

[72]  Martha Boaz,et al.  The Future of Library and Information Science Education , 1978 .

[73]  Abu Saleh Mohammad Mosa,et al.  Building consensus toward a national nursing home information technology maturity model , 2019, J. Am. Medical Informatics Assoc..

[74]  Young-Hee Noh A Korean Study on Verifying Evaluation Indicators and Developing Evaluation Scales for Electronic Resources Using a Pilot Evaluation , 2010 .

[75]  B V Ambrosiadou,et al.  The DELPHI method as a consensus and knowledge acquisition tool for the evaluation of the DIABETES system for insulin administration. , 1999, Medical informatics and the Internet in medicine.

[76]  Lyn Robinson,et al.  Informational balance: slow principles in the theory and practice of information behaviour , 2014, J. Documentation.

[77]  Debbie L. Rabina Reference Materials in LIS Instruction: A Delphi Study. , 2013 .

[78]  Thomas R. Kochtanek,et al.  Delphi Study of Digital Libraries , 1999, Inf. Process. Manag..

[79]  José L. Gascó,et al.  Information systems managers' view about outsourcing in Spain , 2006, Inf. Manag. Comput. Secur..

[80]  Błażej Feret,et al.  The future of academic library and academic librarian : a Delphi study , 1999 .

[81]  Imed Boughzala,et al.  Adaptive use of social networking applications in contemporary organizations: Examining the motivations of Gen Y cohorts , 2016, Int. J. Inf. Manag..

[82]  Julie McLeod,et al.  Consulting records management oracles—a Delphi in practice , 2007 .

[83]  Ling-Feng Hsieh,et al.  Performance evaluation for university electronic libraries in Taiwan , 2006, Electron. Libr..

[84]  Paul Michael Di Gangi,et al.  Exploring the use of the Delphi method in accounting information systems research , 2013, Int. J. Account. Inf. Syst..

[85]  C Hales How should the information needs of the aging be met? A Delphi response. , 1985, The Gerontologist.

[86]  Chaim Zins,et al.  Conceptual approaches for defining data, information, and knowledge , 2007, J. Assoc. Inf. Sci. Technol..

[87]  João Cardoso,et al.  Risk Factors Identification in the Domain of Scientific Data Management: A Generic Assessment Supported by a Delphi Study , 2018, World Digit. Libr..

[88]  F. Hasson,et al.  Research guidelines for the Delphi survey technique. , 2000, Journal of advanced nursing.

[89]  Georg Aichholzer,et al.  The Delphi Method: Eliciting Experts’ Knowledge in Technology Foresight , 2009 .

[90]  Donald L. Pyke,et al.  An evaluation of the forecasting of information processing technology and applications , 1972 .

[91]  Christine Stilwell,et al.  The KwaZulu-Natal school library policy and its feasibility for implementation in the province. , 2013 .

[92]  Björn Niehaves,et al.  Iceberg ahead: On electronic government research and societal aging , 2011, Gov. Inf. Q..

[93]  David Streatfield,et al.  Innovative impact planning and assessment through Global Libraries , 2015 .

[94]  Jean-Noël Ezingeard,et al.  Information Assurance and Corporate Strategy: A Delphi Study of Choices, Challenges, and Developments for the Future , 2011, Inf. Syst. Manag..

[95]  James C. Wetherbe,et al.  Key issues in information systems management , 1987 .

[96]  Gene Rowe,et al.  The Delphi technique: Past, present, and future prospects — Introduction to the special issue☆ , 2011 .

[97]  Melody Madrid,et al.  A study of digital curator competences: A survey of experts , 2013 .

[98]  J. Landeta Current validity of the Delphi method in social sciences , 2006 .

[99]  Susan Starr,et al.  Library as place: results of a delphi study. , 2005, Journal of the Medical Library Association : JMLA.

[100]  Chaim Zins,et al.  Knowledge map of information science , 2007, J. Assoc. Inf. Sci. Technol..

[101]  Qinghua Zhu,et al.  Using a Delphi method and the Analytic Hierarchy Process to evaluate the search engines: A case study on Chinese search engines , 2011, Online Inf. Rev..

[102]  Juri Pill,et al.  The Delphi method: Substance, context, a critique and an annotated bibliography , 1971 .

[103]  Richard P. Smiraglia A Research Agenda for Cataloging: The CCQ Editorial Board Responds to the Year of Cataloging Research , 2010 .

[104]  Tom Addison,et al.  E-commerce project development risks: evidence from a Delphi survey , 2003, Int. J. Inf. Manag..

[105]  Jane Lamothe What keeps CARL directors awake at night , 2013 .

[106]  Sirous Alidousti,et al.  Design of a digital library maturity model (DLMM) , 2018, Electron. Libr..

[107]  Sajed M. Abukhader A taxonomy of the expected roles of librarians towards knowledge management , 2019, Library Management.

[108]  Gordon Welty,et al.  Problems Of Selecting Experts For Delphi Exercises , 1972 .

[109]  Marie T. Ascher,et al.  The new Medical Library Association research agenda: final results from a three-phase Delphi study. , 2012, Journal of the Medical Library Association : JMLA.

[110]  Sinnikka Koskiala,et al.  The Finnish Delphi study: forecasting the extent of information technology use in libraries in 1996 and 2010 , 1989 .

[111]  S. Simon,et al.  Prioritizing Measures of Digital Patient Engagement: A Delphi Expert Panel Study , 2017, Journal of medical Internet research.

[112]  Chaim Zins,et al.  Conceptions of information science , 2007, J. Assoc. Inf. Sci. Technol..

[113]  Chris Buckley,et al.  Delphi: a methodology for preferences more than predictions , 1995 .

[114]  Eddie Halpin,et al.  Measuring the value of public libraries in the digital age: What the power people need to know , 2015, J. Libr. Inf. Sci..

[115]  Younghee Noh A Study on Developing Evaluation Criteria for Electronic Resources in Evaluation Indicators of Libraries , 2010 .

[116]  Benjamin T. Hazen,et al.  Evaluating Adoption of Emerging IT for Corporate IT Strategy: Developing a Model Using a Qualitative Method , 2013, Inf. Syst. Manag..

[117]  Nils Pharo,et al.  Academic Librarians in Data Information Literacy Instruction: A Case Study in Meteorology , 2016, Coll. Res. Libr..

[118]  Marvin A. Jolson,et al.  http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless , 2022 .

[119]  F. Hasson,et al.  A critical review of the Delphi technique as a research methodology for nursing. , 2001, International journal of nursing studies.

[120]  Lynn Westbrook Information access issues for interdisciplinary scholars: Results of a Delphi study on women's studies research , 1997 .

[121]  Gülçin Büyüközkan,et al.  Multi-criteria decision making for e-marketplace selection , 2004, Internet Res..

[122]  Pierre Pluye,et al.  The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers , 2018, Educ. Inf..

[123]  R. D. Needham,et al.  Moving best practice forward: delphi characteristics, advantages, potential problems, and solutions. , 2009 .

[124]  S. Simsova A Delphi survey of comparative librarianship , 1975 .

[125]  K. Al-Busaidi SWOT of social networking sites for group work in government organizations: An exploratory Delphi study from IT managers' perspective , 2014 .

[126]  Heting Chu Research methods in library and information science: A content analysis , 2015 .