Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research

The authors of this article reviewed the methodology of three common consensus methods: nominal group process, consensus development panels, and the Delphi technique. The authors set out to determine how a majority of researchers are conducting these studies, how they are analyzing results, and subsequently the manner in which they are reporting their findings. The authors conclude with a set of guidelines and suggestions designed to aid researchers who choose to use the consensus methodology in their work. Overall, researchers need to describe their inclusion criteria. In addition to this, on the basis of the current literature the authors found that a panel size of 5 to 11 members was most beneficial across all consensus methods described. Lastly, the authors agreed that the statistical analyses done in consensus method studies should be as rigorous as possible and that the predetermined definition of consensus must be included in the ultimate manuscript. More specific recommendations are given for each of the three consensus methods described in the article.

[1]  Helen Sampson,et al.  Useful but no Oracle: reflections on the use of a Delphi Group in a multi-methods policy research study , 2015 .

[2]  D. Weiner,et al.  Essential components of a medical student curriculum on chronic pain management in older adults: results of a modified delphi process. , 2002, Pain medicine.

[3]  P. D. del Nido,et al.  Cardiac magnetic resonance versus routine cardiac catheterization before bidirectional Glenn anastomosis: long-term follow-up of a prospective randomized trial. , 2013, The Journal of thoracic and cardiovascular surgery.

[4]  Chia-Chien Hsu,et al.  The Delphi Technique: Making Sense of Consensus , 2007 .

[5]  N. Black,et al.  An experimental study of determinants of group judgments in clinical guideline development , 2004, The Lancet.

[6]  C. Webb,et al.  The Delphi technique: a methodological discussion. , 1994, Journal of advanced nursing.

[7]  Mark Clayton Delphi: a technique to harness expert opinion for critical decision‐making tasks in education , 1997 .

[8]  Glenn Regehr,et al.  Delphi as a method to establish consensus for diagnostic criteria. , 2003, Journal of clinical epidemiology.

[9]  Peter Y. Chen,et al.  A Consensus Method for Updating Psychosocial Measures Used in NIOSH Health Hazard Evaluations , 2012, Journal of occupational and environmental medicine.

[10]  George Wright,et al.  The Delphi technique as a forecasting tool: issues and analysis , 1999 .

[11]  Suzanne D. Pawlowski,et al.  The Delphi method as a research tool: an example, design considerations and applications , 2004, Inf. Manag..

[12]  Michael Schull,et al.  The development of indicators to measure the quality of clinical care in emergency departments following a modified-delphi approach. , 2002, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[13]  A. Bowling Research Methods in Health , 1998 .

[14]  Fiona Wood,et al.  Keywords in Qualitative Methods: A Vocabulary of Research Concepts , 2006 .

[15]  M. D. de Villiers,et al.  The Delphi technique in health sciences education research , 2005, Medical teacher.

[16]  Aileen Clarke,et al.  Developing a quality criteria framework for patient decision aids: online international Delphi consensus process , 2006, BMJ : British Medical Journal.

[17]  Applications of consensus methods in the improvement of care of paediatric patients: a step forward from a ‘good guess’ , 2013, Acta paediatrica.

[18]  N. Black,et al.  Appropriateness of cholecystectomy in the United Kingdom--a consensus panel approach. , 1991, Gut.

[19]  A. Hutchinson,et al.  Research methods used in developing and applying quality indicators in primary care , 2002, BMJ : British Medical Journal.

[20]  J N Horton,et al.  Nominal group technique , 1980, Anaesthesia.

[21]  Roy C. Schmidt,et al.  MANAGING DELPHI SURVEYS USING NONPARAMETRIC STATISTICAL TECHNIQUES , 1997 .

[22]  P. Davidson,et al.  Using the consensus development conference method in healthcare research. , 2008, Nurse researcher.

[23]  R. Schuler Nominal Group Technique , 2015 .

[24]  D. Furst,et al.  Expert consensus for performing right heart catheterisation for suspected pulmonary arterial hypertension in systemic sclerosis: a Delphi consensus study with cluster analysis , 2013, Annals of the rheumatic diseases.

[25]  C Duffield,et al.  The Delphi technique. , 1988, The Australian journal of advanced nursing : a quarterly publication of the Royal Australian Nursing Federation.

[26]  D. Hunter,et al.  Qualitative Research: Consensus methods for medical and health services research , 1995 .

[27]  J K Anderson,et al.  A forecast of the future for the mental health profession. , 1981, The American psychologist.

[28]  W. E. Riggs,et al.  The delphi technique: A long-range planning tool , 1976 .

[29]  R. Brook,et al.  Consensus methods: characteristics and guidelines for use. , 1984, American journal of public health.

[30]  E A Panacek,et al.  Management of spontaneous pneumothorax: an American College of Chest Physicians Delphi consensus statement. , 2001, Chest.

[31]  J. Landeta Current validity of the Delphi method in social sciences , 2006 .

[32]  Fred Dagenais The Reliability and Convergence of the Delphi Technique. , 1978, The Journal of general psychology.

[33]  Raj Nair,et al.  Methods of formal consensus in classification/diagnostic criteria and guideline development. , 2011, Seminars in arthritis and rheumatism.

[34]  M G Seremetis,et al.  The management of spontaneous pneumothorax. , 1970, Chest.

[35]  Martin Steinert,et al.  A dissensus based online Delphi approach: An explorative research tool , 2009 .