Standing back and looking forward: Editors’ reflections on the 20th Anniversary of Evaluation

For us this is a special issue at a particular moment: the journal Evaluation is marking its 20th anniversary, an occasion that challenges us to stand back and take stock – and to do so in a more nuanced way than the Social Science Citation Index on its own allows.1 It offers an opportunity to reflect on the aspirations that launched Evaluation back in 1995, on where we are now, and where we might be going. As the editors of this special anniversary issue have all been active protagonists since the journal’s launch, we are well placed to give an account of ourselves – even though many of our editorial priorities, developed in dialogue with authors, referees and readers, have only become clear to us through a rear-view mirror.

[1]  Jos Vaessen,et al.  Programme Theory Evaluation, Multicriteria Decision Aid and Stakeholder Values , 2006 .

[2]  A. Gregory.  Problematizing Participation , 2000 .

[3]  Luke Georghiou,et al.  Issues in the Evaluation of Innovation and Technology Policy , 1998 .

[4]  Patricia J. Rogers,et al.  Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions , 2008 .

[5]  Square Pegs and Round Holes , 1999 .

[6]  Petri Virtanen,et al.  Exploring the Missing Links between Cause and Effect , 2004 .

[7]  Gill Callaghan,et al.  Evaluation and Negotiated Order , 2008 .

[8]  Ray Pawson,et al.  Evidence-based Policy: In Search of a Method , 2002 .

[9]  Mhairi Mackenzie,et al.  Theories of Change and Realistic Evaluation , 2007 .

[10]  Peter van der Knaap,et al.  Theory-Based Evaluation and Learning: Possibilities and Challenges , 2004 .

[11]  Julia Espinosa,et al.  Moving towards gender-sensitive evaluation? Practices and challenges in international-development evaluation , 2013 .

[12]  N. Mutrie,et al.  Using programme theory to strengthen research protocol and intervention design within an RCT of a walking intervention , 2013 .

[13]  I. Sanderson,et al.  Evaluation in Complex Policy Systems , 2000 .

[14]  A. Balthasar,et al.  The Effects of Institutional Design on the Utilization of Evaluation , 2006 .

[15]  S. B. Nielsen,et al.  A Nordic evaluation tradition? A look at the peer-reviewed evaluation literature , 2014 .

[16]  Michael Wolf-Branigin,et al.  Using Complexity Theory for Research and Program Evaluation , 2013 .

[17]  Jan-Eric Furubo,et al.  The institutionalization of evaluation matters: Updating the International Atlas of Evaluation 10 years later , 2015 .

[18]  John Mayne,et al.  Contribution analysis: Coming of age? , 2012 .

[19]  Ray Pawson,et al.  Evidence-based Policy: The Promise of `Realist Synthesis' , 2002 .

[20]  Peter van der Knaap.  Policy Evaluation and LearningFeedback, Enlightenment or Argumentation? , 1995 .

[21]  A. Giddens,et al.  Reflexive Modernization: Politics, Tradition and Aesthetics in the Modern Social Order , 1994 .

[22]  Ray Pawson,et al.  Nothing as Practical as a Good Theory , 2003 .

[23]  Nicoletta Stame.  The European Project, Federalism and Evaluation , 2008 .

[24]  Josefien van Olmen,et al.  Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research , 2012 .

[25]  Donald A. Schön,et al.  The Reflective Practitioner: How Professionals Think in Action. , 1987 .

[26]  Eduardo Zapico-Goñi.  Matching Public Management, Accountability and Evaluation in Uncertain Contexts , 2007 .

[27]  Barbara Befani,et al.  Between complexity and generalization: Addressing evaluation challenges with QCA , 2013 .

[28]  S. Staedke,et al.  Strengthening patient-centred communication in rural Ugandan health centres: A theory-driven evaluation within a cluster randomized trial , 2014, Evaluation.

[29]  Ana Manzano-Santaella,et al.  A realistic evaluation of fines for hospital discharges: Incorporating the history of programme evaluations in the analysis , 2011 .

[30]  Giel Ton,et al.  The mixing of methods: A three-step process for improving rigour in impact evaluations , 2012 .

[31]  H. Simons.  Interpret in context: Generalizing from the single case in evaluation , 2015 .

[32]  David Byrne,et al.  Evaluating complex social interventions in a complex world , 2013 .

[33]  Lene Holm Pedersen,et al.  Is Realist Evaluation a Realistic Approach for Complex Reforms? , 2008 .

[34]  Evert Vedung,et al.  Purposes and criteria in network governance evaluation: How far does standard evaluation vocabulary takes us? , 2012 .

[35]  Stephen Martin,et al.  Speeches and Addresses : Evaluation, Inspection and the Improvement Agenda , 2005 .

[36]  Astrid Brousselle,et al.  Evaluating Service Organization Models , 2009, Evaluation.

[37]  Carolyn Kagan,et al.  Critical Connections between Participatory Evaluation, Organizational Learning and Intentional Change in Pluralistic Organizations , 2009 .

[38]  Christopher Pollitt,et al.  The logics of performance management , 2013 .

[39]  Gérard Kœnig.  Realistic Evaluation and Case Studies , 2009 .

[40]  Murray Saunders,et al.  Capturing effects of interventions, policies and programmes in the European context: A social practice perspective , 2011 .

[41]  Nancy Cartwright,et al.  Evidence-Based Policy: A Practical Guide to Doing It Better , 2012 .

[42]  Astrid Brousselle,et al.  Evaluation models and evaluation use , 2012, Evaluation.

[43]  Sandy Oliver,et al.  An Emerging Framework for Including Different Types of Evidence in Systematic Reviews for Public Policy , 2005 .

[44]  J. Lave,et al.  Understanding Practice: Perspectives on Activity and Context , 1996 .

[45]  Burt Perrin,et al.  How to — and How Not to — Evaluate Innovation , 2002 .

[46]  Marian Barnes,et al.  Evidence, Understanding and Complexity , 2003 .

[47]  M. Marra.  Cooperating for a more egalitarian society: Complexity theory to evaluate gender equity , 2015 .

[48]  S. Krupnik.  Evaluating the social mechanism of investment subsidies using an abductive approach , 2012 .

[49]  Michael Woolcock,et al.  Using case studies to explore the external validity of ‘complex’ development interventions , 2013 .

[50]  M. Reynolds.  Equity-focused developmental evaluation using critical systems thinking , 2014 .

[51]  Valérie Pattyn,et al.  Why organizations (do not) evaluate? Explaining evaluation activity through the lens of configurational comparative methods , 2014 .

[52]  Claus C. Rebien.  Participatory Evaluation of Development Assistance , 1996 .

[53]  Peter Dahler-Larsen,et al.  From Programme Theory to Constructivism , 2001 .

[54]  Ville Valovirta.  Evaluation Utilization as Argumentation , 2002 .

[55]  J. Uitto.  Evaluating environment and development: Lessons from international cooperation , 2014 .

[56]  M. Lehtonen,et al.  OECD Environmental Performance Review Programme , 2005 .

[57]  Ray Pawson,et al.  The Science of Evaluation: A Realist Manifesto , 2013 .

[58]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[59]  Benoît Rihoux,et al.  Configurational Comparative Methods , 2011 .

[60]  G. Westhorp,et al.  Using complexity-consistent theory for evaluating complex systems , 2012 .

[61]  Yvonne Kleistra,et al.  Evaluating Diplomacy: A Mission Impossible? , 2010 .

[62]  Mark Kosters,et al.  From mechanism to virtue: Evaluating Nudge theory , 2015 .

[63]  Mita Marra,et al.  Evaluating the Complex: Attribution, Contribution and Beyond , 2017 .

[64]  Per Mickwitz,et al.  A Framework for Evaluating Environmental Policy Instruments Context and Key Concepts , 2003 .

[65]  R. Yin.  Validity and generalization in future case study evaluations , 2013 .

[66]  D. Morgan.  Paradigms Lost and Pragmatism Regained , 2007 .

[67]  Murray Saunders,et al.  The use and usability of evaluation outputs: A social practice approach , 2012 .

[68]  A. Giddens.  Modernity and Self-Identity: Self and Society in the Late Modern Age , 1992, The New Social Theory Reader.

[69]  Denise E. De Souza,et al.  Elaborating the Context-Mechanism-Outcome configuration (CMOc) in realist evaluation: A critical realist perspective , 2013 .

[70]  Tineke A. Abma,et al.  Dealing with Distrust and Power Dynamics: Asymmetric Relations among Stakeholders in Responsive Evaluation , 2010 .

[71]  Tim Blackman,et al.  Using Qualitative Comparative Analysis to understand complex policy problems , 2013 .

[72]  Souraya Sidani,et al.  Quantitative and qualitative methods:: Is There an Alternative? , 1995 .

[73]  M. Bustelo.  Evaluation of Gender Mainstreaming Ideas from a Meta-Evaluation Study , 2003 .

[74]  Cees Leeuwis,et al.  The need for reflexive evaluation approaches in development cooperation , 2015 .