Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research

Abstract Interdisciplinary and multi-stakeholder research is increasingly being promoted and implemented to enhance understanding of global environment change, identify holistic policy solutions, and assist implementation. These research activities are social processes aiming to enhance the exchange and translation of knowledge. Emphasis on the design and management of knowledge exchange is increasing, but learning about how to do this better is hampered by lack of conceptual development and appropriate methods to evaluate complex and multifaceted knowledge exchange processes. This paper therefore develops principles for the evaluation of knowledge exchange in interdisciplinary, multi-stakeholder environmental change research. The paper is based on an analysis of 135 peer-reviewed evaluations of knowledge exchange from diverse disciplines. The results indicate strong relationships between the field of study (e.g. health care, environmental management), the way knowledge and knowledge exchange were conceptualised and implemented, the approach used for the evaluation, and the outcomes being evaluated. A typology of seven knowledge exchange evaluations is presented to guide discussions about the underlying assumptions of different approaches to knowledge exchange and its evaluation. Five principles for knowledge exchange evaluation are also identified: (i) design for multiple end users; (ii) be explicit about why a particular approach to knowledge exchange is expected to deliver its outcomes; (iii) evaluate diverse outcomes; (iv) use evaluations as part of the process of delivering knowledge exchange; and (v) use mixed methods to evaluate knowledge exchange. We conclude that a catch-all approach to evaluation is neither appropriate nor desirable. Instead, approaches that focus on understanding the underlying processes of knowledge exchange, assess the relative contribution of other factors in shaping outcomes in addition to knowledge exchange, and that involve multiple stakeholders in implementing evaluations, will be the most appropriate for evaluating knowledge exchange in interdisciplinary global environmental change research.

[1]  Robert J. Brulle,et al.  Human behavior and sustainability , 2012 .

[2]  B. Hofer,et al.  Dimensionality and Disciplinary Differences in Personal Epistemology. , 2000, Contemporary educational psychology.

[3]  Ann Zukoski,et al.  Participatory evaluation. What is it? Why do it? What are the challenges? , 2002, Community-based public health policy & practice.

[4]  E. Granek,et al.  Co‐Management Approach to Marine Conservation in Mohéli, Comoros Islands , 2005 .

[5]  Terry Schmidt,et al.  Strategic Project Management Made Simple: Practical Tools for Leaders and Teams , 2009 .

[6]  Jo Rycroft-Malone,et al.  Implementing health research through academic and clinical partnerships: a realistic evaluation of the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) , 2011, Implementation science : IS.

[7]  Peter Rossi.  My Views of Evaluation and Their Origins , 2004 .

[8]  Patrice Garin,et al.  Research, part of a Special Feature on Implementing Participatory Water Management: Recent Advances in Theory, Practice and Evaluation Supporting the Shift from State Water to Community Water: Lessons from a Social Learning Approach to Designing Joint Irrigation Projects in Morocco , 2009 .

[9]  J. Molas-Gallart,et al.  Tracing productive interactions to identify social impacts: an example from the social sciences , 2011 .

[10]  W. Sheate,et al.  Knowledge exchange: a review and research agenda for environmental management , 2012, Environmental Conservation.

[11]  Margaret A. Reams,et al.  Linking actions to outcomes in wetland management: An overview of U.S. state wetland management , 2001, Wetlands.

[12]  I. Fazey.  Resilience and Higher Order Thinking , 2010 .

[13]  Alex Mwaki,et al.  Evaluation of the role of school children in the promotion of point-of-use water treatment and handwashing in schools and households--Nyanza Province, Western Kenya, 2007. , 2010, The American journal of tropical medicine and hygiene.

[14]  E. Ruto,et al.  Stakeholder engagement and knowledge exchange in environmental research. , 2012, Journal of environmental management.

[15]  Amber Wutich,et al.  Spanning Boundaries in an Arizona Watershed Partnership: Information Networks as Tools for Entrenchment or Ties for Collaboration? , 2010 .

[16]  Catherine Allan,et al.  Adaptive Environmental Management: A Practitioner's Guide , 2009 .

[17]  Alexander J. Romiszowski,et al.  Instructional development paradigms , 1997 .

[18]  Christina A. Christie,et al.  Evaluation Roots: Tracing Theorists′ Views and Influences , 2004 .

[19]  A. Kothari,et al.  Lessons from the business sector for successful knowledge management in health care: A systematic review , 2011, BMC health services research.

[20]  Brian Crawford,et al.  Small Scale Fisheries Management: Lessons from Cockle Harvesters in Nicaragua and Tanzania , 2010 .

[21]  M. Juntti,et al.  Evidence, politics and power in public policy for the environment , 2009 .

[22]  Trevor W. Parfitt,et al.  The ambiguity of participation: a qualified defence of participatory development , 2004 .

[23]  Laura R. Meagher,et al.  Flows of knowledge, expertise and influence: a method for assessing policy and practice impacts from social science research , 2008 .

[24]  Nicolás L. Gutiérrez,et al.  Leadership, social capital and incentives promote successful fisheries , 2011, Nature.

[25]  Derek Armitage,et al.  A resilience-based framework for evaluating adaptive co-management: Linking ecology, economics and society in a complex world , 2007 .

[26]  M. J. Werff ten Bosch,et al.  KNOWLEDGE MANAGEMENT FOR LAND DEGRADATION MONITORING AND ASSESSMENT: AN ANALYSIS OF CONTEMPORARY THINKING , 2013 .

[27]  M. Reed.  Stakeholder participation for environmental management: A literature review , 2008 .

[28]  Lois-ellin Datta,et al.  Evaluation Theory, Models, and Applications, by Daniel L. Stufflebeam and Anthony J. Shinkfield. San Francisco: Jossey-Bass, 2007. 768 pp. $70.00 , 2007 .

[29]  Arturo Molina,et al.  A knowledge-based development model: the research chair strategy , 2009, J. Knowl. Manag..

[30]  Kara L. Hall,et al.  The science of team science: overview of the field and introduction to the supplement. , 2008, American journal of preventive medicine.

[31]  Douglas P Gross,et al.  Evaluation of a knowledge translation initiative for physical therapists treating patients with work disability , 2009, Disability and rehabilitation.

[32]  M. Albornoz.  Re-Thinking Science: Knowledge and the Public in an Age of Uncertainty , 2003 .

[33]  W. Mauser,et al.  Transdisciplinary global change research: The co-creation of knowledge for sustainability , 2013 .

[34]  Johnny Sung,et al.  The Career Choices and Impact of PhD Graduates in the UK: A Synthesis Review Report prepared for the Economic and Social Research Council (ESRC) "Science in Society" Team and the Research Councils UK (RCUK) Research Careers and Diversity Unit , 2009 .

[35]  Mike Walsh,et al.  Key characteristics of knowledge transfer and exchange in healthcare: integrative literature review. , 2011, Journal of advanced nursing.

[36]  D. Roux,et al.  Framework for participative reflection on the accomplishment of transdisciplinary research programs , 2010 .

[37]  J A Bellamy,et al.  A systems approach to the evaluation of natural resource management initiatives. , 2001, Journal of environmental management.

[38]  James Perry,et al.  Evaluation of an educational program for the caregivers of persons diagnosed with a malignant glioma. , 2007, Canadian oncology nursing journal = Revue canadienne de nursing oncologique.

[39]  Robert A. Francis,et al.  Post-normal science and the art of nature conservation , 2010 .

[40]  R. Christley,et al.  A cluster-randomised controlled trial to compare the effectiveness of different knowledge-transfer interventions for rural working equid users in Ethiopia. , 2011, Preventive veterinary medicine.

[41]  R. Kasperson,et al.  Sustainability Science , 2019, Critical Skills for Environmental Professionals.

[42]  David Manuel-Navarrete,et al.  You say you want a revolution? Transforming education and capacity building in response to global change , 2013 .

[43]  Grace Warner,et al.  Advancing coordinated care in four provincial healthcare systems: evaluating a knowledge-exchange intervention. , 2011, Healthcare policy = Politiques de sante.

[44]  L. Bracken,et al.  Making sense of policy implementation: the construction and uses of expertise and evidence in managing freshwater environments , 2013 .

[45]  P. Heylings,et al.  Evaluating governance: A process for understanding how co-management is functioning, and why, in the Galapagos Marine Reserve , 2007 .

[46]  Louisa Evans,et al.  Assessing the impact of fisheries co-management interventions in developing countries: a meta-analysis. , 2011, Journal of environmental management.

[47]  Thomas D. Sisk,et al.  A Framework to Evaluate Ecological and Social Outcomes of Collaborative Management: Lessons from Implementation with a Northern Arizona Collaborative Group , 2010, Environmental management.

[48]  Alison E Powell,et al.  Communicating Social Research Findings More Effectively: What Can We Learn from Other Fields?. , 2012 .

[49]  Sarah Bowen,et al.  Demystifying knowledge translation: learning from the community , 2005, Journal of health services research & policy.

[50]  Sharon E Straus,et al.  Knowledge translation is the use of knowledge in health care decision making. , 2011, Journal of clinical epidemiology.

[51]  Thaddeus R. Miller,et al.  Epistemological Pluralism: Reorganizing Interdisciplinary Research , 2008 .

[52]  Boyd Swinburn,et al.  An exploratory cluster randomised controlled trial of knowledge translation strategies to support evidence-informed decision-making in local governments (The KT4LG study) , 2011, BMC public health.

[53]  Richard B. Norgaard,et al.  Learning and knowing collectively , 2004 .

[54]  Robyn Tamblyn,et al.  Promoting Evidence-Based Health Policy, Programming, and Practice for Seniors: Lessons from a National Knowledge Transfer Project* , 2003, Canadian Journal on Aging / La Revue canadienne du vieillissement.

[55]  Jane Dixon,et al.  Applying logics of change to the evaluation of community development in health promotion , 1994 .

[56]  Mathias Beike,et al.  Evaluation Roots Tracing Theorists Views And Influences , 2016 .

[57]  Sarah Morton.  Exploring and assessing social research impact : a case study of a research partnership’s impacts on policy and practice , 2012 .

[58]  David M. Fetterman,et al.  Empowerment evaluation principles in practice , 2005 .

[59]  Jana M. McPherson,et al.  Ten years of adaptive community-governed conservation: evaluating biodiversity protection and poverty alleviation in a West African hippopotamus reserve , 2010, Environmental Conservation.

[60]  C. Pahl-Wostl,et al.  A conceptual framework for analysing adaptive capacity and multi-level learning processes in resource governance regimes , 2009 .

[61]  Andrea A. Anderson,et al.  The Community Builder's Approach to Theory of Change , 2006 .

[62]  Diane D. Chapman,et al.  New Learning: a different way of approaching conference evaluation , 2007 .

[63]  Jerome K. Vanclay,et al.  Social learning: a knowledge and capacity building approach for adaptive co-management of contested landscapes , 2011 .

[64]  Jerry Ravetz.  Science and Governance in the European Union. A Contribution to the Debate. , 2000 .

[65]  Murali Raman,et al.  Knowledge Management for Social Workers Involved in Disaster Planning and Response in Malaysia: An Action Research Approach , 2011 .

[66]  Gordon Clark,et al.  Students as Transferors of Knowledge , 2008 .

[67]  Sarah Bowen,et al.  A model for collaborative evaluation of university-community partnerships , 2006, Journal of Epidemiology and Community Health.

[68]  M. Kirshbaum.  Translation to practice: a randomised, controlled study of an evidence-based booklet for breast-care nurses in the United Kingdom. , 2008, Worldviews on evidence-based nursing.

[69]  C. Alvarado,et al.  Knowledge systems, health care teams, and clinical practice: a study of successful change , 2010, Advances in health sciences education : theory and practice.

[70]  Joyce E Wilkinson,et al.  Introducing the Learning Practice--I. The characteristics of Learning Organizations in Primary Care. , 2004, Journal of evaluation in clinical practice.

[71]  John Gabbay,et al.  A Case Study of Knowledge Management in Multiagency Consumer-Informed `Communities of Practice': Implications for Evidence-Based Policy Development in Health and Social Services , 2003 .

[72]  Michele Grimaldi,et al.  Knowledge-based inter-organizational collaborations , 2010, J. Knowl. Manag..

[73]  Judith E. Innes,et al.  Outcomes of Collaborative Water Policy Making: Applying Complexity Thinking to Evaluation , 2001 .

[74]  Richard Margoluis,et al.  A Systematic Test of an Enterprise Strategy for Community‐Based Biodiversity Conservation , 2001 .

[75]  Ioan Fazey,et al.  Integrating local and scientific knowledge for environmental management. , 2010, Journal of environmental management.

[76]  I. Nonaka,et al.  SECI, Ba and Leadership: a Unified Model of Dynamic Knowledge Creation , 2000 .

[77]  Kuan Yew Wong,et al.  Knowledge management performance measurement: A review , 2011 .

[78]  Judith E. Innes,et al.  Consensus Building and Complex Adaptive Systems , 1999 .

[79]  Mita Marra,et al.  Knowledge partnerships for development: what challenges for evaluation? , 2004 .

[80]  Annette Boaz,et al.  Assessing the impact of research on policy: a literature review , 2009 .

[81]  A. Kothari,et al.  Indicators at the interface: managing policymaker-researcher collaboration , 2011 .

[82]  S. Funtowicz,et al.  Science for the Post-Normal Age , 1993, Commonplace.

[83]  Jorge Higinio Maldonado,et al.  Evaluating the role of co-management in improving governance of marine protected areas: an experimental approach in the Colombian Caribbean. , 2010 .

[84]  Annette Boaz,et al.  Real-world approaches to assessing the impact of environmental research on policy , 2011 .