Data Analysis and Synthesis Within a Realist Evaluation: Toward More Transparent Methodological Approaches

Realist evaluations are increasingly used in the study of complex health interventions. The methodological procedures applied within realist evaluations however are often inexplicit, prompting scholars to call for increased transparency and more detailed description within realist studies. This publication details the data analysis and synthesis process used within two realist evaluation studies of community health interventions taking place across Uganda, Tanzania, and Kenya. Using data from several case studies across all three countries and the data analysis software NVivo, we describe in detail how data were analyzed and subsequently synthesized to refine middle-range theories. We conclude by discussing the strengths and weaknesses of the approach taken, providing novel methodological recommendations. The aim of providing this detailed descriptive account of the analysis and synthesis in these two studies is to promote transparency and contribute to the advancement of realist evaluation methodologies.

[1]  R. Pawson,et al.  Theory-driven evaluation of public health programmes , 2009 .

[2]  Kathy V. C. Parris Implementing Accountability , 1996, IEEE Softw..

[3]  Ray Pawson,et al.  RAMESES publication standards: realist syntheses , 2013, BMC Medicine.

[4]  B. Marchal,et al.  Towards Developing an Initial Programme Theory: Programme Designers and Managers Assumptions on the Antiretroviral Treatment Adherence Club Programme in Primary Health Care Facilities in the Metropolitan Area of Western Cape Province, South Africa , 2016, PloS one.

[5]  A. Manzano,et al.  The craft of interviewing in realist evaluation , 2016 .

[6]  Andrew Georgiou,et al.  A realist evaluation of the role of communities of practice in changing healthcare practice , 2011, Implementation science : IS.

[7]  E. Toomey,et al.  Adapting health interventions for local fit when scaling-up: a realist review protocol , 2019, BMJ Open.

[8]  Mansoor A. F. Kazi,et al.  Realist evaluation in practice , 2003 .

[9]  B. Marchal,et al.  How to develop a theory-driven evaluation design? Lessons learned from an adolescent sexual and reproductive health programme in West Africa , 2010, BMC public health.

[10]  G. Moore,et al.  Integration of Motivational Interviewing into Practice in the National Exercise Referral Scheme in Wales: A Mixed Methods Study , 2011, Behavioural and Cognitive Psychotherapy.

[11]  Pieterbas Lalleman,et al.  Realist Evaluation , 2019, TVZ - Verpleegkunde in praktijk en wetenschap.

[12]  Bart Criel,et al.  Advancing the application of systems thinking in health: a realist evaluation of a capacity building programme for district managers in Tumkur, India , 2014, Health Research Policy and Systems.

[13]  Josefien van Olmen,et al.  Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research , 2012 .

[14]  A. Hewison Evidence-Based Policy , 2008, Policy, politics & nursing practice.

[15]  Peter Kamuzora,et al.  Implementing accountability for reasonableness framework at district level in Tanzania: a realist evaluation , 2011, Implementation science : IS.

[16]  R. Merton Social Theory and Social Structure , 1958 .

[17]  E. Waters,et al.  'Motor magic' : evaluation of a community capacity-building approach to supporting the development of preschool children (part 2) , 2006 .

[18]  Anita Kothari,et al.  Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review , 2014, Implementation Science.

[19]  Joanne Greenhalgh,et al.  RAMESES II reporting standards for realist evaluations , 2016, BMC Medicine.

[20]  Debbie Tolson,et al.  Developing a managed clinical network in palliative care: a realistic evaluation. , 2007, International journal of nursing studies.

[21]  Pierre Pluye,et al.  Assessing the outcomes of participatory research: protocol for identifying, selecting, appraising and synthesizing the literature for realist review , 2011, Implementation science : IS.

[22]  B. Marchal,et al.  Turning around an ailing district hospital: a realist evaluation of strategic changes at Ho Municipal Hospital (Ghana) , 2010, BMC public health.

[23]  N. Tumwesigye,et al.  How do community health committees contribute to capacity building for maternal and child health? A realist evaluation protocol , 2016, BMJ Open.

[24]  Joyce E. Wilkinson,et al.  Collective action for implementation: a realist evaluation of organisational collaboration in healthcare , 2015, Implementation Science.

[25]  B. Marchal,et al.  Realist evaluation of the antiretroviral treatment adherence club programme in selected primary healthcare facilities in the metropolitan area of Western Cape Province, South Africa: a study protocol , 2016, BMJ Open.

[26]  T. Greenhalgh,et al.  Uncovering the Benefits of Participatory Research: Implications of a Realist Review for Health Research and Practice , 2012, The Milbank quarterly.

[27]  D. Anthony Evidence-based Policy: A Realist Perspective , 2007 .

[28]  Graham Moore,et al.  Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions , 2016, Evaluation.

[29]  Richard Byng,et al.  Using Realistic Evaluation to Evaluate a Practice-level Intervention to Improve Primary Healthcare for Patients with Long-term Mental Illness , 2005 .

[30]  M. San Sebastiån,et al.  Mechanisms for achieving adolescent-friendly services in Ecuador: a realist evaluation approach , 2012, Global health action.

[31]  Jo Rycroft-Malone,et al.  A realistic evaluation: the case of protocol-based care , 2010, Implementation science : IS.