Evaluating participatory research: Framework, methods and implementation results.

This paper describes a structured participatory process and associated evaluation protocol developed to detect systems learning by decision makers involved in the management of natural resources. A series of facilitated participatory workshops were conducted to investigate learning when decision makers and influencers were confronted with the multiple, complex interactions arising from decisions concerned with the nexus of water, food and energy security. The participatory process and evaluation of learning were trialled in the Greater Mekong Subregion (GMS), where integrated scientific evidence was systematically presented to challenge existing beliefs concerned with the effectiveness of proposed policy actions and development investments. Consistent with theoretical propositions, individually held values, beliefs and attitudes were deployed as the primary factors (and psychometrics) that underpin and influence environmental management decision making. Observed and statistically significant changes in the three psychometrics expressed by decision makers in response to the facilitated presentation of scientific evidence during the participatory process, provided supportive evidence of systems learning and the evaluation protocol.

[1]  S. Schwartz,et al.  Toward A Universal Psychological Structure of Human Values , 1987 .

[2]  I. Ajzen.  The theory of planned behavior , 1991 .

[3]  D. Jonassen.  Evaluating constructivistic learning , 1991 .

[4]  Pieter W. G. Bots,et al.  A Framework for Clarifying “Participation” in Participatory Research to Prevent its Rejection for the Wrong Reasons , 2010 .

[5]  Thomas Dietz,et al.  A Brief Inventory of Values , 1998 .

[6]  Annette Boaz,et al.  Assessing the impact of research on policy: A review of the literature for a project on bridging research and policy through outcome evaluation , 2008 .

[7]  Alex Smajgl,et al.  Visions, beliefs, and transformation: exploring cross-sector and transboundary dynamics in the wider Mekong region , 2015 .

[8]  C. Chess.  Evaluating Environmental Public Participation: Methodological Questions , 2000 .

[9]  I. Ajzen,et al.  Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research , 1977 .

[10]  M. Conner,et al.  The Theory of Planned Behaviour , 2004 .

[11]  G. Nigel Gilbert,et al.  Agent-Based Models , 2007 .

[12]  Liana J. Williams,et al.  Remotely engaged? Towards a framework for monitoring the success of stakeholder engagement in remote regions , 2010 .

[13]  Dwight Barry,et al.  A Science for Survival: Values and Conservation Biology , 1996 .

[14]  Judith Petts,et al.  Evaluating the Effectiveness of Deliberative Processes: Waste Management Case-studies , 2001 .

[15]  Judith E. Innes,et al.  Outcomes of Collaborative Water Policy Making: Applying Complexity Thinking to Evaluation , 2001 .

[16]  Amy R. Poteete,et al.  Working Together: Collective Action, the Commons, and Multiple Methods in Practice , 2010 .

[17]  Lee Roy Beach,et al.  Image theory: an Alternative to Normative Decision theory , 1993 .

[18]  Matthias Wolff,et al.  Using Traditional Ecological Knowledge to Improve Holistic Fisheries Management: Transdisciplinary Modeling of a Lagoon Ecosystem of Southern Mexico , 2013 .

[19]  Derek Armitage,et al.  A resilience-based framework for evaluating adaptive co-management: Linking ecology, economics and society in a complex world , 2007 .

[20]  Shalom H. Schwartz,et al.  A test of a model for reducing measured attitude-behavior discrepancies. , 1972 .

[21]  P. K. Thornton,et al.  Methodological challenges in evaluating impact of crop-livestock interventions , 2004 .

[22]  S. Schwartz.  Universals in the Content and Structure of Values: Theoretical Advances and Empirical Tests in 20 Countries , 1992 .

[23]  François Bousquet,et al.  Modelling with stakeholders , 2010, Environ. Model. Softw..

[24]  M. Etienne.  Companion Modelling: A Participatory Approach to Support Sustainable Development , 2010 .

[25]  Brian Castellani,et al.  Sociology and Complexity Science , 2009 .

[26]  Pascal Perez,et al.  Assessment and Monitoring of the Effects of the ComMod Approach , 2014 .

[27]  Thomas Dietz,et al.  Values and vegetarianism : an exploratory analysis , 2010 .

[28]  J. Pedersen,et al.  Habermas' Method: Rational Reconstruction , 2008 .

[29]  A. Smajgl,et al.  Accounting for multiple governance scales in integrated modelling and decision support. , 2010 .

[30]  S. Page,et al.  The emergence of zoning policy games in exurban jurisdictions: Informing collective action theory , 2009 .

[31]  K. Beratan.  A Cognition-based View of Decision Processes in Complex Social–Ecological Systems , 2007 .

[32]  J. Habermas.  Moral Consciousness and Communicative Action , 1990 .

[33]  S. Funtowicz,et al.  EMERGENT COMPLEX SYSTEMS , 1994 .

[34]  P. Todd,et al.  Simple Heuristics That Make Us Smart , 1999 .

[35]  Stephen Kaplan,et al.  New Ways to Promote Proenvironmental Behavior: Human Nature and Environmentally Responsible Behavior , 2000 .

[36]  I. Ajzen.  Attitudes, Personality and Behavior , 1988 .

[37]  Stephen Kaplan,et al.  Human nature and environmentally responsible behavior , 2000 .

[38]  R. Pielke,et al.  The neglected heart of science policy: reconciling supply of and demand for science , 2007 .

[39]  Steven C Bankes,et al.  Tools and techniques for developing policies for complex and uncertain systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Alex Smajgl,et al.  The Water-Food-Energy Nexus in the Mekong Region , 2013 .

[41]  Troy D. Abel,et al.  A Value-Belief-Norm Theory of Support for Social Movements: The Case of Environmentalism , 1999 .

[42]  Alexander Smajgl,et al.  Challenging beliefs through multi-level participatory modelling in Indonesia , 2010, Environ. Model. Softw..

[43]  Jürgen Habermas,et al.  Concluding Comments on Empirical Approaches to Deliberative Politics , 2005 .

[44]  Alex Smajgl,et al.  A framework to bridge science and policy in complex decision making arenas , 2013 .

[45]  S. Kellert,et al.  Community Natural Resource Management: Promise, Rhetoric, and Reality , 2000 .

[46]  S. Schwartz,et al.  Toward a theory of the universal content and structure of values: Extensions and cross-cultural replications. , 1990 .

[47]  Christina von Haaren,et al.  Social learning can benefit decision-making in landscape planning: Gartow case study on climate change adaptation, Elbe valley biosphere reserve , 2012 .

[48]  Fabio Boschetti,et al.  Can we learn how complex systems work , 2011 .

[49]  Natalie A. Jones,et al.  Evaluating Participatory Modeling: Developing a Framework for Cross-Case Analysis , 2009, Environmental management.

[50]  Peter M. Todd,et al.  Fast and frugal heuristics for environmentally bounded minds , 2001 .

[51]  Eric Kemp-Benedict,et al.  Developing detailed foresight narratives: a participatory technique from the Mekong region , 2013 .

[52]  J. Sterman.  Risk Communication on Climate: Mental Models and Mass Balance , 2008, Science.