Facilitating comparative effectiveness research in cancer genomics: evaluating stakeholder perceptions of the engagement process.

AIMS The Center for Comparative Effectiveness Research in Cancer Genomics completed a 2-year stakeholder-guided process for the prioritization of genomic tests for comparative effectiveness research studies. We sought to evaluate the effectiveness of engagement procedures in achieving project goals and to identify opportunities for future improvements. MATERIALS & METHODS The evaluation included an online questionnaire, one-on-one telephone interviews and facilitated discussion. Responses to the online questionnaire were tabulated for descriptive purposes, while transcripts from key informant interviews were analyzed using a directed content analysis approach. RESULTS A total of 11 out of 13 stakeholders completed both the online questionnaire and interview process, while nine participated in the facilitated discussion. Eighty-nine percent of questionnaire items received overall ratings of agree or strongly agree; 11% of responses were rated as neutral with the exception of a single rating of disagreement with an item regarding the clarity of how stakeholder input was incorporated into project decisions. Recommendations for future improvement included developing standard recruitment practices, role descriptions and processes for improved communication with clinical and comparative effectiveness research investigators. CONCLUSIONS Evaluation of the stakeholder engagement process provided constructive feedback for future improvements and should be routinely conducted to ensure maximal effectiveness of stakeholder involvement.

[1]  D. Cook,et al.  Users' guides to the medical literature: XXIII. Qualitative research in health care A. Are the results of the study valid? Evidence-Based Medicine Working Group. , 2000, JAMA.

[2]  S. Greenfield,et al.  Comparative Effectiveness Research: A Report From the Institute of Medicine , 2009, Annals of Internal Medicine.

[3]  S. Tunis,et al.  How best to engage patients, doctors, and other stakeholders in designing comparative effectiveness studies. , 2010, Health affairs.

[4]  M. Patton Qualitative research and evaluation methods , 1980 .

[5]  G. Rowe,et al.  Public Participation Methods: A Framework for Evaluation , 2000 .

[6]  G. Guyatt,et al.  Users' guides to the medical literature. , 1993, JAMA.

[7]  L. Bogart,et al.  Community-based participatory research: partnering with communities for effective and sustainable behavioral health interventions. , 2009, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[8]  John W. Creswell,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2010 .

[9]  Hsiu-Fang Hsieh,et al.  Three Approaches to Qualitative Content Analysis , 2005, Qualitative health research.

[10]  David M. Konisky,et al.  Values, Conflict, and Trust in Participatory Environmental Planning , 2000 .

[11]  Thomas Webler,et al.  “Right” Discourse in Citizen Participation: An Evaluative Yardstick , 1995 .

[12]  Jeanne-Marie Guise,et al.  Engaging Stakeholders To Identify and Prioritize Future Research Needs , 2011 .

[13]  T. A. Lee,et al.  Prioritizing Comparative‐Effectiveness Research Topics via Stakeholder Involvement: An Application in COPD , 2011, Clinical pharmacology and therapeutics.

[14]  J. Boote,et al.  What does it mean to involve consumers successfully in NHS research? A consensus study , 2004, Health expectations : an international journal of public participation in health care and health policy.

[15]  Gene Rowe,et al.  Stakeholder engagement in food risk management , 2011 .

[16]  Douglas K. Martin,et al.  Evaluating priority setting success in healthcare: a pilot study , 2010, BMC health services research.

[17]  Jason Chilvers,et al.  Upping the ante: A conceptual framework for designing and evaluating participatory technology assessments , 2006 .

[18]  Helen R. Bayliss,et al.  The PIRICOM study : a systematic review of the conceptualisation, measurement, impact and outcomes of patients and public involvement in health and social care research , 2010 .

[19]  K. Chalkidou,et al.  Comparative effectiveness research and evidence-based health policy: experience from four countries. , 2009, The Milbank quarterly.

[20]  J. Eyles,et al.  Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. , 2003, Social science & medicine.

[21]  Roy Marsh,et al.  Evaluation of a Deliberative Conference , 2004 .

[22]  David Veenstra,et al.  How comparative effectiveness research can help advance 'personalized medicine' in cancer treatment. , 2011, Health affairs.

[23]  Patti Delger,et al.  Community-Based Participatory Research: A Review of the Literature With Strategies for Community Engagement , 2009, Journal of developmental and behavioral pediatrics : JDBP.

[24]  Ortwin Renn A Model for an Analytic−Deliberative Process in Risk Management , 1999 .

[25]  J. Boote,et al.  Consumer involvement in health research: a review and research agenda. , 2002, Health policy.

[26]  Rahber Thariani,et al.  Prioritization in Comparative Effectiveness Research: The CANCERGEN Experience , 2012, Medical care.

[27]  Scott D Ramsey,et al.  Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. , 2012, Journal of comparative effectiveness research.

[28]  John N Lavis,et al.  "It all depends": conceptualizing public involvement in the context of health technology assessment agencies. , 2010, Social science & medicine.

[29]  Rosemary Barber,et al.  Can the impact of public involvement on research be evaluated? A mixed methods study , 2012, Health expectations : an international journal of public participation in health care and health policy.