The MEPPP Framework: A Framework for Monitoring and Evaluating Participatory Planning Processes

Evaluating participatory processes, participatory planning processes especially, can be challenging. Due to their complexity, these processes require a specific approach to evaluation. This paper proposes a framework for evaluating projects that have adopted a participatory planning approach: the monitoring and evaluation of participatory planning processes (MEPPP) framework. The MEPPP framework is applied to one case study, a participatory planning process in the Rwenzori region in Uganda. We suggest that this example can serve as a guideline for researchers and practitioners to set up the monitoring and evaluation of their participatory planning process of interest by following six main phases: (1) description of the case, (2) clarification of the M&E viewpoint(s) and definition of the M&E objective(s), (3) identification of the context, process and outputs/outcomes analytical variables, (4) development of the M&E methods and data collection, (5) data analysis, and (6) sharing of the M&E results. Results of the application of the MEPPP framework in Uganda demonstrate the ability of the framework to tackle the complexity of participatory planning processes. Strengths and limitations of the MEPPP framework are also discussed.

[1]  Kristen Purcell,et al.  Public Participation and the Environment: Do We Know What Works? , 1999 .

[2]  Ronald D. Brunner.  Context-sensitive monitoring and evaluation for the World Bank , 2004 .

[3]  Judith E. Innes,et al.  Consensus Building and Complex Adaptive Systems , 1999 .

[4]  Emily Talen,et al.  Do Plans Get Implemented? A Review of Evaluation in Planning , 1996 .

[5]  A. Hatchuel.  Towards an epistemology of collective action: management research as a responsive and actionable discipline , 2005 .

[6]  M. Maarleveld,et al.  Demystifying Facilitation in Participatory Development , 2000 .

[7]  Michael Quinn Patton,et al.  Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use , 2010 .

[8]  J. Tiefenbacher,et al.  Evaluating the Effectiveness of Public Participation Efforts by Environmental Agencies: Repermitting a Smelter in El Paso, Texas, USA , 2008 .

[9]  Katherine A. Daniell,et al.  Co-Engineering and Participatory Water Management: Organisational Challenges for Water Governance , 2018 .

[10]  J A Bellamy,et al.  A systems approach to the evaluation of natural resource management initiatives. , 2001, Journal of environmental management.

[11]  G. Rowe,et al.  Public Participation Methods: A Framework for Evaluation , 2000 .

[12]  Karen Bickerstaff,et al.  Participatory Local Governance and Transport Planning , 2001 .

[13]  Rita G. O'Sullivan,et al.  Evaluation Voices: Promoting evaluation from within programs through collaboration , 1998 .

[14]  Robert Y. Cavana,et al.  Towards a new framework for evaluating systemic problem structuring methods , 2013, Eur. J. Oper. Res..

[15]  J. Fereday,et al.  Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development , 2006 .

[16]  Angela Oels.  Evaluating Stakeholder Dialogues , 2006 .

[17]  John Mingers,et al.  Critical Heuristics of Social Planning: A New Approach to Practical Philosophy , 1983 .

[18]  T. Webler,et al.  Fairness and competence in citizen participation : evaluating models for environmental discourse , 1995 .

[19]  Wei Li,et al.  Institutions, Institutional Change, and Economic Performance , 2009, SSRN Electronic Journal.

[20]  Thomas C. Beierle.  Democracy in Practice: Public Participation in Environmental Decisions , 2010 .

[21]  E. Ostrom.  A General Framework for Analyzing Sustainability of Social-Ecological Systems , 2009, Science.

[22]  H. van Vlaenderen,et al.  Evaluating participation processes in community development , 1995 .

[23]  Heike Schroeder,et al.  Institutions and Environmental Change: Principal Findings, Applications, and Research Frontiers , 2008 .

[24]  A. George,et al.  Case Studies and Theory Development in the Social Sciences , 2005 .

[25]  Kathleen E. Halvorsen,et al.  Assessing Public Participation Techniques for Comfort, Convenience, Satisfaction, and Deliberation , 2001, Environmental management.

[26]  M. Reed.  Stakeholder participation for environmental management: A literature review , 2008 .

[27]  John Forester,et al.  The Deliberative Practitioner: Encouraging Participatory Planning Processes , 1999 .

[28]  Paul Burton,et al.  How Would We Know What Works? , 2006 .

[29]  L. Finlay.  “Outing” the Researcher: The Provenance, Process, and Practice of Reflexivity , 2002, Qualitative health research.

[30]  G. Rowe,et al.  Evaluating Public Participation Exercises: Strategic and Practical Issues , 2005 .

[31]  J. Glicken.  Getting stakeholder participation ‘right’: a discussion of participatory processes and possible pitfalls , 2000 .

[32]  P. Sabatier.  An advocacy coalition framework of policy change and the role of policy-oriented learning therein , 1988 .

[33]  Sten Jönsson.  Institutions and Organizations , 1997 .

[34]  Michael Lehning,et al.  Enabling effective problem-oriented research for sustainable development , 2012 .

[35]  Elke Herrfahrdt-Pähle,et al.  Continuity and Change in Social-ecological Systems: the Role of Institutional Resilience , 2012 .

[36]  T. Sager.  Communicative Planning Theory , 1994 .

[37]  David M. Konisky,et al.  Values, Conflict, and Trust in Participatory Environmental Planning , 2000 .

[38]  R. Margerum.  Collaborative Planning , 2002 .

[39]  P. Healey.  Collaborative Planning in Perspective , 2003 .

[40]  Donna Harawy.  Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective , 2022, Philosophical Literary Journal Logos.

[41]  Seth Tuler,et al.  Unlocking the Puzzle of Public Participation , 2002 .

[42]  S. Farolfi,et al.  WAT-A-GAME: sharing water and policies in your own basin , 2009 .

[43]  James R. Sanders,et al.  تقويم البرنامج : طرق بديلة و إرشادات عملية = Program evaluation Alternative Approaches and Practical Guidelines , 1987 .

[44]  E. Ostrom.  Understanding Institutional Diversity , 2005 .

[45]  John M. Wilson,et al.  The impact of contingency factors on validation of problem structuring methods , 2010, J. Oper. Res. Soc..

[46]  P. Lascoumes,et al.  Sociologie de l'action publique , 2007 .

[47]  Katherine E. Ryan,et al.  Exploring evaluator role and identity , 2002 .

[48]  Gustavo Stubrich.  The Fifth Discipline: The Art and Practice of the Learning Organization , 1993 .

[49]  Michael Doyle,et al.  Facilitator's Guide to Participatory Decision-Making , 1998 .

[50]  M. P. Walsh,et al.  Systemic evaluation: a participative, multi-method approach , 2007, J. Oper. Res. Soc..

[51]  Michael Bennett,et al.  Toward an Empowerment Zone Evaluation , 1999 .

[53]  Pierre Lefèvre,et al.  Comprehensive Participatory Planning and Evaluation (CPPE) , 2011 .

[54]  Archon Fung.  Varieties of Participation in Complex Governance , 2006 .

[55]  M. Douglas.  How Institutions Think , 1986 .

[56]  Henry Mintzberg.  The rise and fall of strategic planning , 1993 .

[57]  O. Barreteau,et al.  Drivers of Environmental Institutional Dynamics in Decentralized African Countries , 2015, Environmental Management.

[58]  G. Midgley.  Dealing with coercion: Critical Systems Heuristics and beyond , 1997 .

[59]  J. Norberg,et al.  ADAPTIVE GOVERNANCE OF SOCIAL-ECOLOGICAL SYSTEMS , 2005 .

[60]  M. Patton,et al.  Enhancing the quality and credibility of qualitative analysis. , 1999, Health services research.

[61]  Orville F. Poland,et al.  Implementation & Evaluation , 1982, Polity.

[62]  Marvin C. Alkin,et al.  Using Evaluations: Does Evaluation Make a Difference? , 1979 .

[63]  Barbara Pini.  On Being a Nice Country Girl and an Academic Feminist: Using Reflexivity in Rural Social Research. , 2004 .

[64]  Richard Warren Smith,et al.  A theoretical basis for participatory planning , 1973 .

[65]  Roy Marsh,et al.  Evaluation of a Deliberative Conference , 2004 .

[66]  Gerald Midgley,et al.  Systems thinking for evaluation , 2007 .

[67]  John Mayne,et al.  Contribution analysis: Coming of age? , 2012 .

[68]  James Mahoney,et al.  Process Tracing and Historical Explanation , 2015 .

[69]  T. A..  “Sociologie de l'action publique“, Pierre Lascoumes et Patrick... , 2014 .

[70]  David Byrne,et al.  Evaluating complex social interventions in a complex world , 2013 .

[71]  Milbrey W. McLaughlin,et al.  Evaluation and education : at quarter century , 1992 .

[72]  David R. Godschalk,et al.  Malking Waves: Public Participation in State Water Planningi , 1981 .

[73]  Bronwyn L Horsey,et al.  Developing and applying a framework to evaluate participatory research for sustainability , 2007 .

[74]  Judith E. Innes,et al.  Outcomes of Collaborative Water Policy Making: Applying Complexity Thinking to Evaluation , 2001 .

[75]  B. Cooke,et al.  Participation: the New Tyranny? , 2001 .

[76]  J. Greene.  Evaluation as Advocacy , 1997 .

[77]  Oran R. Young,et al.  Institutions and Environmental Change: The Scientific Legacy of a Decade of IDGEC Research , 2008 .

[78]  Elinor Ostrom,et al.  A Framework to Analyze the Robustness of Social-ecological Systems from an Institutional Perspective , 2004 .

[79]  I. Sanderson,et al.  Evaluation in Complex Policy Systems , 2000 .

[80]  J. B. Rosener,et al.  Citizen Participation: Can We Measure Its Effectiveness? , 1978 .

[81]  W. Scott,et al.  Institutions and Organizations. , 1995 .

[82]  Judith Petts,et al.  Evaluating the Effectiveness of Deliberative Processes: Waste Management Case-studies , 2001 .

[83]  G. Rowe,et al.  Evaluating Public-Participation Exercises: A Research Agenda , 2004 .

[84]  Pieter W. G. Bots,et al.  A Framework for Clarifying “Participation” in Participatory Research to Prevent its Rejection for the Wrong Reasons , 2010 .

[85]  Pat Ricia,et al.  Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions , 2008 .

[86]  Alexander Conley,et al.  Evaluating Collaborative Natural ResourceManagement , 2003 .

[87]  H. Simon,et al.  Strategy and organizational evolution , 1993 .

[88]  G. Widdershoven,et al.  Participation of marginalized groups in evaluation: mission impossible? , 2010, Evaluation and program planning.

[89]  Nils Ferrand,et al.  Wat-A-Game, a toolkit for building role-playing games about integrated water management , 2012 .

[90]  Lucie Laurian,et al.  Evaluation of Public Participation , 2009 .

[91]  Miri Levin-Rozalis,et al.  Searching for the Unknowable: A Process of Detection — Abductive Research Generated by Projective Techniques , 2004 .

[92]  T. Sager.  Evaluation Methods in Local Participatory Planning , 1981 .

[93]  Sandy Taut,et al.  What have we learned about stakeholder involvement in program evaluation , 2008 .

[94]  B. Williams.  Prosaic or Profound? The Adoption of Systems Ideas by Impact Evaluation , 2015 .

[95]  B. Lankford.  Integrated, adaptive and domanial water resources management , 2008 .

[96]  John Ward,et al.  Towards understanding participatory processes: Framework, application and results. , 2015, Journal of environmental management.

[97]  Ralph Renger,et al.  Systemic Evaluation, Impact Evaluation and Logic Models , 2011 .

[98]  T. Webler,et al.  Fairness and Competence in Citizen Participation , 1995 .

[99]  Martin Onyach-Olaa.  The challenges of implementing decentralisation: recent experiences in Uganda , 2003 .

[100]  Liisa Tyrväinen,et al.  A framework for evaluating and designing collaborative planning , 2013 .

[101]  M. Etienne.  Companion Modelling: A Participatory Approach to Support Sustainable Development , 2010 .

[102]  Pascal Perez,et al.  Assessment and Monitoring of the Effects of the ComMod Approach , 2014 .