Using a Complex Adaptive Systems Perspective to Illuminate the Concept of Evaluation Capacity Building in a Network

While evaluation capacity building (ECB) may hold promise for fostering evaluation, little is known about how it is operationalized within a network. This article presents initial findings from a National Science Foundation–funded research project (Complex Adaptive Systems as a Model for Network Evaluations) that used concepts from complex adaptive systems theory to develop case studies of ECB within the Nanoscale Informal Science Education Network. The project used a multiple case study approach to explore ECB within four Network workgroups. Cross-case themes documented characteristics of the system and ECB within it. Evaluation capacity was evident in several ways, including people's comfort with evaluation, evaluation-related skills, evaluation processes used, and the value placed on evaluation. Ultimately, the study identified several complex adaptive system features that fostered Network ECB: massive entanglement and neighbor interactions, information flow/feedback loops, stability and flexibility/adaptability, redundancy and diversity of evaluative skills and knowledge, and both centralized and distributed control.

[1]  J. Cousins,et al.  Organizational Capacity To Do and Use Evaluation: Results of a Pan-Canadian Survey of Evaluators , 2009, Canadian Journal of Program Evaluation.

[2]  J. Bradley Cousins,et al.  Utilization Effects of Participatory Evaluation , 2003 .

[3]  J. Cousins,et al.  Framing the Capacity to Do and Use Evaluation , 2014 .

[4]  Jane Whynot,et al.  Application of an organizational evaluation capacity self-assessment instrument to different organizations: similarities and lessons learned. , 2015, Evaluation and program planning.

[5]  Jennifer Brown Urban,et al.  Relationships Between Quantitative Measures of Evaluation Plan and Program Model Quality and a Qualitative Measure of Participant Perceptions of an Evaluation Capacity Building Approach , 2015 .

[6]  Susan N. Labin,et al.  Developing Common Measures in Evaluation Capacity Building , 2014 .

[7]  Heather H. Boyd,et al.  Evaluation capacity building in complex organizations , 2008 .

[8]  Engaging stakeholders in the planning of a collaborative multi-agency evaluation: The HousingPlus Collaborative Communities Project , 2008 .

[9]  J. Horn.  Human Research and Complexity Theory , 2008 .

[10]  J. King.  Making sense of participatory evaluation practice , 1998 .

[11]  Tamsin Haggis.  ‘Knowledge Must Be Contextual’: Some possible implications of complexity and dynamic systems theories for educational research , 2008 .

[12]  M. Mason.  What Is Complexity Theory and What Are Its Implications for Educational Change? , 2008 .

[13]  Stacie A. Toal,et al.  The unique character of involvement in multi-site evaluation settings. , 2009, Evaluation and program planning.

[14]  Mary E. Arnold,et al.  Developing Evaluation Capacity in Extension 4-H Field Faculty , 2006 .

[15]  Frances P Lawrenz,et al.  Collaborative evaluation communities in urban schools: A model of evaluation capacity building for STEM education , 2006 .

[16]  Hallie Preskill,et al.  A Multidisciplinary Model of Evaluation Capacity Building , 2008 .

[17]  Melissa Dark,et al.  A comparison of linear and systems thinking approaches for program evaluation illustrated using the Indiana Interdisciplinary GK-12. , 2009, Evaluation and program planning.

[18]  Dennis Sumara,et al.  Challenging images of knowing: complexity science and educational research , 2005 .

[19]  Abraham Wandersman,et al.  A Research Synthesis of the Evaluation Capacity Building Literature , 2012 .

[20]  J. Cousins,et al.  Understanding Dimensions of Organizational Evaluation Capacity , 2013 .

[21]  D. Compton,et al.  Toward a definition of the ECB process: A conversation with the ECB literature , 2002 .

[22]  Frances P Lawrenz,et al.  A Collaborative Immersio Approach to Evaluation Capacity Building , 2008 .

[23]  Sebastian Lemire,et al.  Measuring Evaluation Capacity—Results and Implications of a Danish Study , 2011 .

[24]  F. Balcazar,et al.  Understanding and Measuring Evaluation Capacity , 2013 .

[25]  J. King,et al.  Exploring Organizational Evaluation Capacity and Evaluation Capacity Building , 2017 .

[26]  Marie Connolly,et al.  Developing Evaluation Cultures in Human Service Organisations , 2013 .

[27]  An assessment scale of ECB: perspectives from the faculty and staff of technical colleges in Taiwan. , 2011, Evaluation and program planning.

[28]  Grant Blashki,et al.  Building evaluation capacity: definitional and practical implications from an Australian case study. , 2007, Evaluation and program planning.

[29]  Alan Boyle,et al.  International Law and Sustainable Development , 1999 .

[30]  Jean A. King.  Getting people involved: The origin of interactive evaluation , 2012 .

[31]  Patricia J. Rogers,et al.  Teaching People to Fish? Building the Evaluation Capability of Public Sector Organizations , 2003 .