Formative Evaluation and Complex Health Improvement Initiatives: A Learning System to Improve Theory, Implementation, Support, and Evaluation

Sustainable community health improvement often requires the implementation of complex interventions in complex systems. Drawing from the Four Keys to Success frame (theory, implementation, support, and evaluation), this article describes how we used a formative evaluation approach to foster a learning system capable of monitoring and addressing emerging community needs within the Spreading Community Accelerators Through Learning and Evaluation (SCALE) initiative—a national capacity-building effort to support 24 community coalitions’ progress toward a Culture of Health. The formative evaluation approach resulted in critical advancements to the theory, implementation, and nature of supports provided in SCALE. These improvements enabled the SCALE evaluation team to shift from the initial focus on program implementation issues to a greater emphasis on downstream factors (community-level outcomes). The ability of formative evaluation to grapple with the emerging challenges of implementing complex interventions in complex systems makes it particularly valuable for community health improvement initiatives.

[1]  Michael W. Painter,et al.  Aligning Forces for Quality: a program to improve health and health care in communities across the United States. , 2008, Health affairs.

[2]  G. Wrenn,et al.  The Readiness for Integrated Care Questionnaire (RICQ): An Instrument to Assess Readiness to Integrate Behavioral Health and Primary Care , 2017, The American journal of orthopsychiatry.

[3]  R. Aungst,et al.  Healthy People 2020 , 2013, American Journal of Kidney Diseases.

[4]  Beth Siegel,et al.  Multisector Partnerships Need Further Development To Fulfill Aspirations For Transforming Regional Health And Well-Being. , 2018, Health affairs.

[5]  D. Finegood,et al.  Getting from analysis to action: framing obesity research, policy and practice with a solution-oriented complex systems lens. , 2008, HealthcarePapers.

[6]  Abraham Wandersman,et al.  The Quality Implementation Framework: A Synthesis of Critical Steps in the Implementation Process , 2012, American journal of community psychology.

[7]  A. Wandersman Four Keys to Success (Theory, Implementation, Evaluation, and Resource/System Support): High Hopes and Challenges in Participation , 2009, American journal of community psychology.

[8]  P. Rossi,et al.  Evaluation: A systematic approach, 5th ed. , 1989 .

[9]  J. Durlak,et al.  Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation , 2008, American journal of community psychology.

[10]  A. Plough Building a culture of health: a critical role for public health services and systems research. , 2015, American journal of public health.

[11]  Eric K Van Beurden,et al.  Making sense in a complex landscape: how the Cynefin Framework from Complex Adaptive Systems Theory can inform health promotion practice. , 2013, Health promotion international.

[12]  D. Snowden,et al.  A leader's framework for decision making , 2007 .

[13]  P. Sandercock,et al.  Framework for design and evaluation of complex interventions to improve health , 2000, BMJ : British Medical Journal.

[14]  C. Achilles,et al.  Evaluation: A Systematic Approach , 1980 .

[15]  Andrew V. Dane,et al.  Program integrity in primary and early secondary prevention: are implementation effects out of control? , 1998, Clinical psychology review.

[16]  K. Dooley,et al.  Health Care Organizations as Complex Adaptive Systems , 2003 .

[17]  Abraham Wandersman,et al.  A formative multi-method approach to evaluating training. , 2016, Evaluation and program planning.

[18]  Dawn K Wilson,et al.  Updating the FORECAST formative evaluation approach and some implications for ameliorating theory failure, implementation failure, and evaluation failure. , 2013, Evaluation and program planning.

[19]  Michael Quinn Patton,et al.  Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use , 2010 .

[20]  Abbie Brown,et al.  Design experiments: Theoretical and methodological challenges in creating complex interventions in c , 1992 .

[21]  Melinda M Davis,et al.  Learning Evaluation: blending quality improvement and implementation research methods to study healthcare innovations , 2015, Implementation Science.

[22]  A. Darzi,et al.  Systematic review of the application of the plan–do–study–act method to improve quality in healthcare , 2013, BMJ quality & safety.

[23]  Karen A. Blase,et al.  Implementation Research: A Synthesis of the , 2005 .

[24]  Gareth Parry,et al.  Increasing the generalisability of improvement research with an improvement replication programme , 2011, Quality and Safety in Health Care.

[25]  Allan Collins,et al.  Design Research: Theoretical and Methodological Issues , 2004 .

[26]  Allan Collins,et al.  Toward a Design Science of Education , 1992 .

[27]  Candice Bowman,et al.  The role of formative evaluation in implementation research and the QUERI experience , 2006, Journal of General Internal Medicine.

[28]  Abraham Wandersman,et al.  Toward an Evidence-Based System for Innovation Support for Implementing Innovations with Quality: Tools, Training, Technical Assistance, and Quality Assurance/Quality Improvement , 2012, American journal of community psychology.

[29]  Anne C. Kubisch,et al.  Voices From the Field III: Lessons and Challenges for Foundations Based on Two Decades of Community-Change Efforts , 2011 .

[30]  S. Syme,et al.  The importance of community development for health and well-being , 2009 .