Research Synthesis Methods in an Age of Globalized Risks: Lessons from the Global Burden of Foodborne Disease Expert Elicitation

We live in an age that increasingly calls for national or regional management of global risks. This article discusses the contributions that expert elicitation can bring to efforts to manage global risks and identifies challenges faced in conducting expert elicitation at this scale. In doing so it draws on lessons learned from conducting an expert elicitation as part of the World Health Organizations (WHO) initiative to estimate the global burden of foodborne disease; a study commissioned by the Foodborne Disease Epidemiology Reference Group (FERG). Expert elicitation is designed to fill gaps in data and research using structured, transparent methods. Such gaps are a significant challenge for global risk modeling. Experience with the WHO FERG expert elicitation shows that it is feasible to conduct an expert elicitation at a global scale, but that challenges do arise, including: defining an informative, yet feasible geographical structure for the elicitation; defining what constitutes expertise in a global setting; structuring international, multidisciplinary expert panels; and managing demands on experts’ time in the elicitation. This article was written as part of a workshop, “Methods for Research Synthesis: A Cross‐Disciplinary Approach” held at the Harvard Center for Risk Analysis on October 13, 2013.

[1]  James K Hammitt,et al.  Introduction to the Special Series on Research Synthesis: A Cross-Disciplinary Approach. , 2015, Risk analysis : an official publication of the Society for Risk Analysis.

[2]  N. Speybroeck,et al.  Calculating disability-adjusted life years to quantify burden of disease , 2014, International Journal of Public Health.

[3]  Valentina Bosetti,et al.  Facing the Experts: Survey Mode and Expert Elicitation , 2014 .

[4]  Yael Grushka-Cockayne,et al.  Trimmed Opinion Pools and the Crowd's Calibration Problem , 2012, Manag. Sci..

[5]  Shahram Sarkani,et al.  The effect of the number of seed variables on the performance of Cooke′s classical model , 2014, Reliab. Eng. Syst. Saf..

[6]  David R. Mandel,et al.  Probabilistic Coherence Weighting for Optimizing Expert Forecasts , 2013, Decis. Anal..

[7]  N. Speybroeck,et al.  WHO Initiative to Estimate the Global Burden of Foodborne Diseases , 2013, The Lancet.

[8]  Willy P Aspinall,et al.  An expert judgement assessment of future sea level rise from the ice sheets , 2013 .

[9]  Sara M Pires,et al.  Assessing the applicability of currently available methods for attributing foodborne disease to sources, including food and food commodities. , 2013, Foodborne pathogens and disease.

[10]  Alan D. Lopez,et al.  Common values in assessing health outcomes from disease and injury: disability weights measurement study for the Global Burden of Disease Study 2010 , 2012, The Lancet.

[11]  J. Garrett Morris,et al.  Annual cost of illness and quality-adjusted life year losses in the United States due to 14 foodborne pathogens. , 2012, Journal of food protection.

[12]  Roger M. Cooke,et al.  Expert judgement and re-elicitation for prion disease risk uncertainties , 2012 .

[13]  XIII Services of the Centers for Disease Control and Prevention (CDC) , 2012, Red Book (2012).

[14]  Shi-Woei Lin,et al.  Effects of overconfidence and dependence on aggregated probability judgments , 2012 .

[15]  J. Utzinger,et al.  Global burden of human food-borne trematodiasis: a systematic review and meta-analysis. , 2012, The Lancet. Infectious diseases.

[16]  Franco Flandoli,et al.  Comparison of a new expert elicitation model with the Classical Model, equal weights and single experts, using a cross-validation technique , 2011, Reliab. Eng. Syst. Saf..

[17]  H. Carabin,et al.  Clinical Manifestations Associated with Neurocysticercosis: A Systematic Review , 2011, PLoS neglected tropical diseases.

[18]  M. Widdowson,et al.  Foodborne Illness Acquired in the United States—Major Pathogens , 2011, Emerging infectious diseases.

[19]  R. Tauxe,et al.  Foodborne Illness Acquired in the United States—Unspecified Agents , 2011, Emerging infectious diseases.

[20]  Judith Rousseau,et al.  Combining expert opinions in prior elicitation , 2010 .

[21]  H. Carabin,et al.  A Systematic Review of the Frequency of Neurocyticercosis with a Focus on People with Epilepsy , 2010, PLoS neglected tropical diseases.

[22]  D. Sack,et al.  Etiology of Diarrhea in Older Children, Adolescents and Adults: A Systematic Review , 2010, PLoS neglected tropical diseases.

[23]  S. Hoffmann,et al.  Food Safety and Risk Governance in Globalized Markets , 2010, Health matrix.

[24]  W. Aspinall.  A route to more tractable expert advice , 2010, Nature.

[25]  J. Desenclos,et al.  RAISIN - a national programme for early warning, investigation and surveillance of healthcare-associated infection in France. , 2009, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[26]  C. Stein,et al.  Estimating the global burden of foodborne diseases--a collaborative effort. , 2009, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[27]  Estimating the global burden of foodborne diseases: a collaborative effort. , 2009, Releve epidemiologique hebdomadaire.

[28]  W. Aspinall,et al.  Emergency planning and mitigation at Vesuvius: A new evidence-based approach , 2008 .

[29]  林希偉,et al.  Can Cooke's Model Sift Out Better Experts and Produce Well-Calibrated Aggregated Probabilities , 2008 .

[30]  Shi-Woei Lin,et al.  Can Cooke’s model sift out better experts and produce well-calibrated aggregated probabilities? , 2008, 2008 IEEE International Conference on Industrial Engineering and Engineering Management.

[31]  Roger M Cooke,et al.  Attribution of foodborne pathogens using structured expert elicitation. , 2008, Foodborne pathogens and disease.

[32]  Roger M. Cooke,et al.  Response to discussants , 2008, Reliab. Eng. Syst. Saf..

[33]  Roger M. Cooke,et al.  TU Delft expert judgment data base , 2008, Reliab. Eng. Syst. Saf..

[34]  Robert T. Clemen,et al.  Comment on Cooke's classical method , 2008, Reliab. Eng. Syst. Saf..

[35]  Vicki M. Bier,et al.  A study of expert overconfidence , 2008, Reliab. Eng. Syst. Saf..

[36]  A. Prüss-Ustün,et al.  The Global Burden of Disease Assessments—WHO Is Responsible? , 2007, PLoS neglected tropical diseases.

[37]  R. L. Winkler,et al.  Advances in Decision Analysis: Aggregating Probability Distributions , 2007 .

[38]  S. Berg.  Snowball Sampling—I , 2006 .

[39]  H. J. van der Fels-Klerx,et al.  A Structured Expert Judgment Study for a Model of Campylobacter Transmission During Broiler‐Chicken Processing , 2005, Risk analysis : an official publication of the Society for Risk Analysis.

[40]  A. Bryman,et al.  Encyclopedia of Social Science Research Methods , 2004 .

[41]  Majid Ezzati,et al.  For Personal Use. Only Reproduce with Permission from the Lancet Publishing Group , 2022 .

[42]  C. Murray,et al.  Global burden of disease , 1997, The Lancet.

[43]  R. Cooke.  Experts in Uncertainty: Opinion and Subjective Probability in Science , 1991 .