Beyond the “Toolkit Approach”: Arts Impact Evaluation Research and the Realities of Cultural Policy‐Making

This article presents a reflection on the possibility and potential advantages of the development of a humanities‐based approach to assessing the impact of the arts, which attempts to move away from a paradigm of evaluation based on a one‐size‐fits‐all model usually reliant on empirical methodologies borrowed from the social sciences. A “toolkit approach” to arts impact assessment, as the article argues, demands excessive simplifications, and its popularity is linked to its perceived advocacy potential rather than to any demonstrable contribution it may make to a genuine understanding of the nature and potential effects of artistic engagement. The article also explores the relationship between research, advocacy and the actual realities of policy‐making with a view to proposing a critical research agenda for impact evaluation based on Carol Weiss’s notion of the “enlightenment” function of policy‐oriented research. In particular, the article attempts to highlight the contribution that cultural policy scholars working within the humanities could make to this area of policy research.

[1]  Carol H. Weiss,et al.  Chapter 3 The haphazard connection: Social science and public policy , 1995 .

[2]  Eleanor Chelimsky,et al.  The Purposes of Evaluation in a Democratic Society , 2006 .

[3]  Eleonora Belfiore,et al.  RETHINKING THE SOCIAL IMPACTS OF THE ARTS , 2007 .

[4]  L. Kriesberg.  Local government and the arts. , 1983 .

[5]  F. Fischer.  Reframing Public Policy: Discursive Politics and Deliberative Practices , 2003 .

[6]  M. Bérubé.  The Utility of the Arts and Humanities , 2003 .

[7]  Michael Paraskos.  WHAT GOOD ARE THE ARTS , 2006 .

[8]  Eleonora Belfiore,et al.  Determinants of Impact: Towards a Better Understanding of Encounters with the Arts , 2007 .

[9]  T. Jowell.  Government and the value of culture. , 2004 .

[10]  P. Merli,et al.  Evaluating the social impact of participation in arts activities , 2002 .

[11]  Kathryn E. Newcomer,et al.  Government As Structural Context for Evaluation , 2006 .

[12]  I. Sanderson.  Is it ‘what works’ that matters? Evaluation and evidence‐based policy‐making , 2003 .

[13]  J. Geoghegan.  Return to Reason! , 1942, Nature.

[14]  S. Nutley,et al.  Evidence and the policy process , 2000 .

[15]  J. Sterne.  Cultural Policy Studies and the Problem of Political Representation , 2002 .

[16]  Eleonora Belfiore,et al.  Researching the social impact of the arts: literature, fiction and the novel , 2009 .

[17]  C. Madden.  Indicators for arts and cultural policy: A Global perspective , 2005 .

[18]  S. Galloway.  Theory-based evaluation and the social impact of the arts , 2009 .

[19]  A. Edgar,et al.  Need humanities be so useless? Justifying the place and role of humanities as a critical resource for performance and practice , 2006, Medical Humanities.

[20]  R. Lowe.  Policy Bureaucracy: Government with a Cast of Thousands , 2009 .

[21]  Beryl A. Radin,et al.  Beyond Machiavelli: Policy Analysis Comes of Age , 2000 .

[22]  P. U. Hohendahl,et al.  The Future of the Research University and the Fate of the Humanities , 2005 .

[23]  J. Ahearne.  Notes from a French perspective , 2006 .

[24]  Eleonora Belfiore.  Art as a means of alleviating social exclusion: Does it really work? A critique of instrumental cultural policies and social impact studies in the UK , 2002 .

[25]  M. Rothbard.  The hermeneutical invasion of philosophy and economics , 1989 .

[26]  Eleonora Belfiore,et al.  The social impact of the arts : an intellectual history , 2008 .

[27]  P. Wells.  New Labour and Evidence Based Policy Making , 2005 .

[28]  S. Toulmin.  Return to Reason , 2001 .

[29]  David O'Brien,et al.  Measuring the Value of Culture , 2010 .

[30]  Ralph Hambrick,et al.  Rethinking Democratic Accountability , 2002 .

[31]  F. Puffelen.  Abuses of conventional impact studies in the arts , 1996 .

[32]  Peter Wells,et al.  New Labour and evidence based policy making: 1997-2007 , 2007 .

[33]  J. Caust.  Putting the “art” back into arts policy making: how arts policy has been “captured” by the economists and the marketers , 2003 .

[34]  Peter C Smith,et al.  What works?Evidence-based policy and practice in public services , 2000 .

[35]  Eleonora Belfiore.  The social impacts of the arts – myth or reality? , 2006 .

[36]  L. Menand,et al.  Dangers Within and Without , 2005 .

[37]  Jamie Cowling.  For Art's Sake?: Society and the Arts in the 21st Century , 2004 .

[38]  Xiaoying Wang.  Farewell to the Humanities , 2005, Rethinking Marxism.

[39]  G. Haber.  For Art's Sake , 1975 .

[40]  Wayne Parsons,et al.  From Muddling Through to Muddling Up - Evidence Based Policy Making and the Modernisation of British Government , 2002 .

[41]  V. Hodgkinson,et al.  Measuring the Impact of the Nonprofit Sector , 2002 .

[42]  Cultural economics and cultural policy: A discussion in the Danish context , 1995 .

[43]  Carol H. Weiss,et al.  The haphazard connection: social science and public policy , 1995 .

[44]  Sandra Nutley,et al.  Editorial: What Works? The Role of Evidence in Public Sector Policy and Practice , 1999 .

[45]  Beryl A. Radin,et al.  Challenging the Performance Movement: Accountability, Complexity, and Democratic Values , 2006 .

[46]  E. Page,et al.  Policy Bureaucracy: Government with a Cast of Thousands , 2005 .

[47]  Sara Selwood,et al.  The politics of data collection: Gathering, analysing and using data about the subsidised cultural sector in England , 2002 .

[48]  Paul DiMaggio.  Measuring the Impact of the Nonprofit Sector on Society Is Probably Impossible but Possibly Useful , 2002 .

[49]  Michael Hill,et al.  The policy process in the modern state , 1997 .

[50]  I. Sanderson.  Evaluation, Policy Learning and Evidence-Based Policy Making , 2002 .

[51]  S. Toulmin.  Cosmopolis: The Hidden Agenda of Modernity , 1990 .

[52]  Imre Szeman.  Culture and Globalization, or, The Humanities in Ruins , 2003 .

[53]  Frank Fischer,et al.  Reframing Public Policy , 2003 .

[54]  M. Finlayson.  Making Sense of New Labour , 2003 .

[55]  Laura C. Leviton,et al.  Research On the Utilization of Evaluations , 1981 .