Assessing the impact of research on policy: a literature review

Understanding the impact of research on policy is a vital, and often overlooked, element of policy-making. A systematic literature review was conducted to examine methods for evaluating the impact of research on policy outcomes. The review focused in particular on strategic policy levels (rather than implementation) and waste, environment and pollution policy. The review draws on an international literature, although it is limited to English language publications. The findings identify the different methods used, the advantages and disadvantages of different approaches and the methods that are most effective (particularly in terms of cost). The field of research impact evaluation is currently experiencing rapid development, and there is scope to develop and apply new conceptual frameworks and innovative methods for evaluation, and to conduct and publish more research impact evaluations.

[1]  Jenny M. Lewis.  A network approach for researching partnerships in health , 2005, Australia and New Zealand health policy.

[2]  M. Bamberger,et al.  Monitoring and evaluation : some tools, methods, and approaches , 2004 .

[3]  Jennie Popay,et al.  How are policy makers using evidence? Models of research utilisation and local NHS policy making , 2000, Journal of epidemiology and community health.

[4]  Douglas R. Williams,et al.  Measuring the economic benefits of research and development: The current state of the art , 1998 .

[5]  D. Zilberman,et al.  The Value of Economic Research , 1997 .

[6]  Evan S. Michelson.  Approaches to research and development performance assessment in the United States: An analysis of recent evaluation trends , 2006 .

[7]  Jennifer Rubin,et al.  Policy and practice impacts of research funded by the Economic and Social Research Council , 2007 .

[8]  F. Carden,et al.  Learning from complexity: The International Development Research Centre's experience with Outcome Mapping , 2002 .

[9]  Steven Wooding,et al.  Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's 'Arthritis Research Campaign' , 2004, Health research policy and systems.

[10]  J. Holland,et al.  Social development as knowledge building: research as a sphere of policy influence , 2003 .

[11]  Ronald N. Kostoff,et al.  Assessing research impact: US government retrospective and quantitative approaches , 1994 .

[12]  F. Carden.  Issues in assessing the policy influence of research , 2004 .

[13]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[14]  Annette Boaz,et al.  Assessing the impact of research on policy: A review of the literature for a project on bridging research and policy through outcome evaluation , 2008 .

[15]  A. Porter,et al.  A societal outcomes map for health research and policy. , 2006, American journal of public health.

[16]  H. Cameron,et al.  RTD Evaluation Toolbox - Assessing The Socio-Economic Impact of RTD Policies , 2002 .

[17]  L. Butler,et al.  Metrics or Peer Review? Evaluating the 2001 UK Research Assessment Exercise in Political Science , 2009 .

[18]  D. Rider-Smith,et al.  Institutionalising impact orientation: building a performance management approach that enhances the impact orientation of research organisations , 2002 .

[19]  L. Kalucy,et al.  UNDERSTANDING & MEASURING RESEARCH IMPACT , 2005 .

[20]  M. Buxton,et al.  Assessing benefits from health research projects: can we use questionnaires instead of case studies? , 1999 .

[21]  V. Smith.  Measuring the benefits of social science research , 1998 .

[22]  Jordi Molas-Gallart,et al.  Policy and Practice Impacts of ESRC Funded Research Case Study of the ESRC Centre for Business Research , 2007 .

[23]  T. Luukkonen-Gronow,et al.  Scientific research evaluation: a review of methods and various contexts of their application , 1987 .

[24]  D. Romney,et al.  Informal Traders Lock Horns with the Formal Milk Industry: The role of research in pro-poor dairy policy shift in Kenya , 2006 .

[25]  Anton J. Nederhof,et al.  Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review , 2006, Scientometrics.

[26]  A. Devaux,et al.  Horizontal Evaluation: Stimulating social learning among peers , 2006 .

[27]  Ronald N. Kostoff,et al.  Federal research impact assessment: Axioms, approaches, applications , 1995, Scientometrics.

[28]  Nicholas Mays,et al.  Describing the impact of health services and policy research , 2007, Journal of health services research & policy.

[29]  H. Davies,et al.  Using Evidence: How research can inform public services , 2007 .

[30]  Sharon A. Jones,et al.  Evaluating the Science-Policy Interface for Climate Change Research , 1999 .

[31]  Pauline E. Ginsberg,et al.  Foundations of program evaluation: Theories of practice , 1993 .

[32]  G. Norton,et al.  Policy for plenty: measuring the benefits of policy-oriented social science research , 1997 .

[33]  Inge van der Weijden,et al.  Evaluation of Research in Context A Quick Scan of an Emerging Field , 2007 .

[34]  Carol H. Weiss.  Evaluation : methods for studying programs and policies , 1997 .

[35]  R. Pielke,et al.  Usable information for policy: An appraisal of the U.S. Global Change Research Program , 1995 .

[36]  Ugur Muldur,et al.  Ex-ante impact assessment of research programmes: The experience of the European Union's 7th Framework Programme , 2007 .

[37]  I. Hovland,et al.  The PRSP Initiative: Multilateral Policy Change and the Role of Research , 2003 .

[38]  John Gabbay,et al.  A Case Study of Knowledge Management in Multiagency Consumer-Informed `Communities of Practice': Implications for Evidence-Based Policy Development in Health and Social Services , 2003 .