Evaluating patient and stakeholder engagement in research: moving from theory to practice.

Despite the growing demand for research that engages stakeholders, there is limited evidence in the literature to demonstrate its value - or return on investment. This gap indicates a general lack of evaluation of engagement activities. To adequately inform engagement activities, we need to further investigate the dividends of engaged research, and how to evaluate these effects. This paper synthesizes the literature on hypothesized impacts of engagement, shares what has been evaluated and identifies steps needed to reduce the gap between engagement's promises and the underlying evidence supporting its practice. This assessment provides explicit guidance for better alignment of engagement's promised benefits with evaluation efforts and identifies specific areas for development of evaluative measures and better reporting processes.

[1]  C. McKevitt,et al.  What is involvement in research and what does it achieve? Reflections on a pilot study of the personal costs of stroke , 2010, Health expectations : an international journal of public participation in health care and health policy.

[2]  C. Dowrick,et al.  Implementation Science Development of a Theory of Implementation and Integration: Normalization Process Theory , 2022 .

[3]  Ken Stein,et al.  A multidimensional conceptual framework for analysing public involvement in health services research , 2008, Health expectations : an international journal of public participation in health care and health policy.

[4]  J. Tritter,et al.  Revolution or evolution: the challenges of conceptualizing patient and public involvement in a consumerist world , 2009, Health expectations : an international journal of public participation in health care and health policy.

[5]  Mary Newburn,et al.  User involvement in the development of a research bid: barriers, enablers and impacts 1 , 2007, Health expectations : an international journal of public participation in health care and health policy.

[6]  N. Britten,et al.  The impact of consumer involvement in research: an evaluation of consumer involvement in the London Primary Care Studies Programme. , 2008, Family practice.

[7]  Jeanne-Marie Guise,et al.  Engaging Stakeholders To Identify and Prioritize Future Research Needs , 2011 .

[8]  J. Boote,et al.  What does it mean to involve consumers successfully in NHS research? A consensus study , 2004, Health expectations : an international journal of public participation in health care and health policy.

[9]  P. Allen-meares,et al.  Using a Collaboratory Model to Translate Social Work Research Into Practice and Policy , 2005 .

[10]  Clive Savory,et al.  Patient and public involvement in translative healthcare research , 2010 .

[11]  M. Ryan,et al.  Involving the public in priority setting: a case study using discrete choice experiments. , 2012, Journal of public health.

[12]  J. Boote,et al.  Public involvement in the systematic review process in health and social care: a narrative review of case examples. , 2011, Health policy.

[13]  M. Greenwood Stakeholder Engagement: Beyond the Myth of Corporate Responsibility , 2007 .

[14]  T. Wykes,et al.  Close to the bench as well as at the bedside: involving service users in all phases of translational research , 2012, Health expectations : an international journal of public participation in health care and health policy.

[15]  Leroi S. Hicks,et al.  A systematic review of community-based participatory research to enhance clinical trials in racial and ethnic minority groups. , 2012, Health services research.

[16]  J. Boote,et al.  Public involvement in the design and conduct of clinical trials: a narrative review of case examples , 2011, Trials.

[17]  A. Palermo,et al.  Community engagement in research: frameworks for education and peer review. , 2010, American journal of public health.

[18]  Julia Abelson,et al.  Assessing the Impacts of Public Participation: Concepts, Evidence and Policy Implications , 2008 .

[19]  Douglas K. Martin,et al.  Priority setting: what constitutes success? A conceptual framework for successful priority setting , 2009, BMC health services research.

[20]  Anne C. Beal,et al.  The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. , 2012, Journal of the American Medical Association (JAMA).

[21]  Helen R. Bayliss,et al.  The PIRICOM study : a systematic review of the conceptualisation, measurement, impact and outcomes of patients and public involvement in health and social care research , 2010 .

[22]  Patricia A Deverka,et al.  Stakeholder engagement in comparative effectiveness research: how will we measure success? , 2012, Journal of comparative effectiveness research.

[23]  K. Facey,et al.  Patients' perspectives in health technology assessment: A route to robust evidence and fair deliberation , 2010, International Journal of Technology Assessment in Health Care.

[24]  Stuart Peacock,et al.  Public participation in health care priority setting: A scoping review. , 2009, Health policy.

[25]  H. Howard Patient and public involvement in health , 2006 .

[26]  J. South,et al.  Participating in community research: exploring the experiences of lay researchers in Bradford , 2009 .

[27]  Rosemary Barber,et al.  Can the impact of public involvement on research be evaluated? A mixed methods study , 2012, Health expectations : an international journal of public participation in health care and health policy.

[28]  L. Forbat,et al.  Involving and engaging patients in cancer and palliative care research: workshop presentation. , 2008, International journal of palliative nursing.

[29]  Erin Holve,et al.  Involving Patients and Consumers in Research: New Opportunities for Meaningful Engagement in Research and Quality Improvement , 2012 .

[30]  G. Rowe,et al.  A Typology of Public Engagement Mechanisms , 2005 .

[31]  Rhonda K. Lewis,et al.  An Interactive and Contextual Model of Community-University Collaborations for Research and Action , 2005, Health education & behavior : the official publication of the Society for Public Health Education.

[32]  W. Cook,et al.  Integrating research and action: a systematic review of community-based participatory research to address health disparities in environmental and occupational health in the USA , 2008, Journal of Epidemiology & Community Health.

[33]  K. Staley,et al.  User involvement leads to more ethically sound research , 2006 .

[34]  J Popay,et al.  Exploring areas of consensus and conflict around values underpinning public involvement in health and social care research: a modified Delphi study , 2014, BMJ Open.

[35]  Hugh Mclaughlin Keeping Service User Involvement in Research Honest , 2010 .

[36]  Michelle B. Leavy,et al.  Incorporating stakeholder perspectives in developing a translation table framework for comparative effectiveness research. , 2012, Journal of comparative effectiveness research.

[37]  Rosemary Barber,et al.  Principles and indicators of successful consumer involvement in NHS research: results of a Delphi study and subgroup analysis. , 2006, Health policy.

[38]  Ann Brown,et al.  The impact of service user involvement in research. , 2005, International journal of health care quality assurance incorporating Leadership in health services.

[39]  A. Booth,et al.  ‘Talking the talk or walking the walk?’ A bibliometric review of the literature on public involvement in health research published between 1995 and 2009 , 2015, Health expectations : an international journal of public participation in health care and health policy.

[40]  B. Happell,et al.  Consumer Participation in Mental Health Research: Articulating a Model to Guide Practice , 2007, Australasian psychiatry : bulletin of Royal Australian and New Zealand College of Psychiatrists.

[41]  L. Leslie,et al.  A Systematic Review of Stakeholder Engagement in Comparative Effectiveness and Patient-Centered Outcomes Research , 2014, Journal of General Internal Medicine.

[42]  Pieter W. G. Bots,et al.  A Framework for Clarifying “Participation” in Participatory Research to Prevent its Rejection for the Wrong Reasons , 2010 .

[43]  J. Sweeney,et al.  Patient and family engagement: a framework for understanding the elements and developing interventions and policies. , 2013, Health affairs.

[44]  K N Lohr,et al.  Community-based participatory research: assessing the evidence. , 2004, Evidence report/technology assessment.

[45]  S. Staniszewska,et al.  Mapping the impact of patient and public involvement on health and social care research: a systematic review , 2014, Health expectations : an international journal of public participation in health care and health policy.

[46]  J. Boote,et al.  Consumer involvement in health research: a review and research agenda. , 2002, Health policy.

[47]  Scott D Ramsey,et al.  Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. , 2012, Journal of comparative effectiveness research.

[48]  H. Hearnshaw,et al.  Assessment of the benefits of user involvement in health research from the Warwick Diabetes Care Research User Group: a qualitative case study , 2007, Health expectations : an international journal of public participation in health care and health policy.

[50]  P. Butow,et al.  Operationalising a model framework for consumer and community participation in health and medical research , 2007, Australia and New Zealand health policy.

[51]  M. Pickersgill Research, engagement and public bioethics: promoting socially robust science , 2011, Journal of Medical Ethics.

[52]  A. Fothergill,et al.  A potential model for the first all Wales mental health service user and carer-led research group. , 2010, Journal of psychiatric and mental health nursing.

[53]  H. Stonnington Nothing About Us Without Us: Disability Oppression and Empowerment , 1998 .

[54]  J. Newton,et al.  Professionals and the public: power or partnership in health research? , 2012, Journal of evaluation in clinical practice.

[55]  G. Widdershoven,et al.  Patients as Partners in Responsive Research: Methodological Notions for Collaborations in Mixed Research Teams , 2009, Qualitative health research.

[56]  Rosemary Barber,et al.  The GRIPP checklist: Strengthening the quality of patient and public involvement reporting in research , 2011, International Journal of Technology Assessment in Health Care.

[57]  S. Tunis,et al.  How best to engage patients, doctors, and other stakeholders in designing comparative effectiveness studies. , 2010, Health affairs.

[58]  K. Carman,et al.  Unresolved tensions in consumer engagement in CER: a US research perspective. , 2013, Journal of comparative effectiveness research.

[59]  T. Williamson,et al.  Impact of public involvement in research on quality of life and society: a case study of research career trajectories , 2010 .

[60]  J. Oetzel,et al.  Process and outcome constructs for evaluating community-based participatory research projects: a matrix of existing measures. , 2012, Health education research.

[61]  A. Howe,et al.  Ensuring public and patient participation in research: a case study in infrastructure development in one UK Research and Development consortium , 2006, Primary Health Care Research & Development.

[62]  Paul Meissner,et al.  A New Taxonomy for Stakeholder Engagement in Patient-Centered Outcomes Research , 2012, Journal of General Internal Medicine.

[63]  V. Montori,et al.  Patient and service user engagement in research: a systematic review and synthesized framework , 2015, Health expectations : an international journal of public participation in health care and health policy.

[64]  J. Eyles,et al.  Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. , 2003, Social science & medicine.

[65]  G. Rowe,et al.  Evaluating Public-Participation Exercises: A Research Agenda , 2004 .

[66]  A. Faulkner,et al.  Evaluating the impact of service user involvement on research: a prospective case study , 2011 .

[67]  P. Beresford The role of service user research in generating knowledge-based health and social care: from conflict to contribution , 2007 .