Integrating impact evaluation in the design and implementation of monitoring marine protected areas

Quasi-experimental impact evaluation approaches, which enable scholars to disentangle effects of conservation interventions from broader changes in the environment, are gaining momentum in the conservation sector. However, rigorous impact evaluation using statistical matching techniques to estimate the counterfactual have yet to be applied to marine protected areas (MPAs). While there are numerous studies investigating ‘impacts’ of MPAs that have generated considerable insights, results are variable. This variation has been linked to the biophysical and social context in which they are established, as well as attributes of management and governance. To inform decisions about MPA placement, design and implementation, we need to expand our understanding of conditions under which MPAs are likely to lead to positive outcomes by embracing advances in impact evaluation methodologies. Here, we describe the integration of impact evaluation within an MPA network monitoring programme in the Bird's Head Seascape, Indonesia. Specifically we (i) highlight the challenges of implementation ‘on the ground’ and in marine ecosystems and (ii) describe the transformation of an existing monitoring programme into a design appropriate for impact evaluation. This study offers one potential model for mainstreaming impact evaluation in the conservation sector.

[1]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[2]  Joachim Claudet,et al.  Improving assessments of marine protected areas , 2010 .

[3]  B. Halpern,et al.  Biological Effects Within No-Take Marine Reserves: A global Synthesis , 2009 .

[4]  Kevin J. Gaston,et al.  The Ecological Performance of Protected Areas , 2008 .

[5]  Stuart Banks,et al.  Global conservation outcomes depend on marine protected areas with five key features , 2014, Nature.

[6]  Andrew Balmford,et al.  Moving Rio Forward and Avoiding 10 More Years with Little Evidence for Effective Conservation Policy , 2014, Conservation biology : the journal of the Society for Conservation Biology.

[7]  A. Underwood.  Beyond BACI: the detection of environmental impacts on populations in the real, but variable, world , 1992 .

[8]  S. Gaines,et al.  Designing marine reserve networks for both conservation and fisheries management , 2010, Proceedings of the National Academy of Sciences.

[9]  R. Rohh.  ALTERNATIVE METHODS FOR EVALUATING THE IMPACT OF INTERVENTIONS An Overview , 2001 .

[10]  Elizabeth R. Selig,et al.  Regional spatio-temporal trends in Caribbean coral reef benthic communities. , 2010 .

[11]  Markku Kanninen,et al.  Evaluating whether protected areas reduce tropical deforestation in Sumatra , 2009 .

[12]  Susana Mourato,et al.  A Global Evaluation of Coral Reef Management Performance: Are MPAs Producing Conservation and Socio-Economic Improvements? , 2011, Environmental management.

[13]  Cindy Huchery,et al.  Global Effects of Local Human Population Density and Distance to Markets on the Condition of Coral Reef Fisheries , 2013, Conservation biology : the journal of the Society for Conservation Biology.

[14]  Hugh P. Possingham,et al.  A comparison of zoning analyses to inform the planning of a marine protected area network in Raja Ampat, Indonesia , 2013 .

[15]  T. Frazer,et al.  Reef fish and habitat relationships in a Caribbean seascape: the importance of reef context , 2007, Coral Reefs.

[16]  Laura B. Rawlings,et al.  Impact Evaluation in Practice: Second Edition , 2010 .

[17]  Alison Green,et al.  Delineating the Coral Triangle , 2009 .

[18]  M. McCormick,et al.  Spatial variability in reef fish distribution, abundance, size and biomass: a multi-scale analysis , 2001 .

[19]  Madeleine C. Bottrill,et al.  Commonalities and complementarities among approaches to conservation monitoring and evaluation , 2014 .

[20]  Rodolphe Devillers,et al.  Reinventing residual reserves in the sea: are we favouring ease of establishment over need for protection? , 2015 .

[21]  Robert J Toonen,et al.  One size does not fit all: the emerging frontier in large-scale marine conservation. , 2013, Marine pollution bulletin.

[22]  Joachim Claudet,et al.  ECOLOGY – Assessing effects of marine protected areas: confounding in space and possible solutions , 2011 .

[23]  Jeremy G. Weber,et al.  Do ICDPs Work? An Empirical Evaluation of Forest-Based Microenterprises in the Brazilian Amazon , 2011, Land Economics.

[24]  Subhrendu K. Pattanayak,et al.  Do Payments for Environmental Services Affect Forest Cover? A Farm-Level Evaluation from Costa Rica , 2012, Land Economics.

[25]  Jasjeet S. Sekhon,et al.  Opiates for the Matches: Matching Methods for Causal Inference , 2009 .

[26]  P. Ferraro,et al.  Measuring the effectiveness of protected area networks in reducing deforestation , 2008, Proceedings of the National Academy of Sciences.

[27]  L. Joppa,et al.  High and Far: Biases in the Location of Protected Areas , 2009, PloS one.

[28]  Paul R. Rosenbaum,et al.  Design Sensitivity and Efficiency in Observational Studies , 2010 .

[29]  Taylor H. Ricketts,et al.  Explaining the global pattern of protected area coverage: relative importance of vertebrate biodiversity, human activities and agricultural suitability , 2008 .

[30]  Benjamin L. Richards,et al.  Environmental Factors Affecting Large-Bodied Coral Reef Fish Assemblages in the Mariana Archipelago , 2012, PloS one.

[31]  J. Lawton,et al.  Rare species, the coincidence of diversity hotspots and conservation strategies , 1993, Nature.

[32]  J. Sanchirico.  Marine Protected Areas as Fishery Policy: A Discussion of the Potential Costs and Benefits , 2000 .

[33]  C. Roberts,et al.  FISHERY BENEFITS OF FULLY PROTECTED MARINE RESERVES: WHY HABITAT AND BEHAVIOR ARE IMPORTANT , 2002 .

[34]  J. Claudeta,et al.  Assessing the effects of marine protected area (MPA) on a reef fish assemblage in a northwestern Mediterranean marine reserve: Identifying community-based indicators , 2006 .

[35]  J. Castilla,et al.  ECOLOGICAL CRITERIA FOR EVALUATING CANDIDATE SITES FOR MARINE RESERVES , 2003 .

[36]  Peter J. Edmunds,et al.  The importance of sampling scale in ecology: kilometer-wide variation in coral reef communities , 1996 .

[37]  M. Tundi Agardy,et al.  Advances in marine conservation: the role of marine protected areas , 1994 .

[38]  Robert L. Pressey,et al.  Evaluating management performance of marine protected area networks in the Philippines , 2014 .

[39]  Russell B. Millar,et al.  Burdens of evidence and the benefits of marine reserves: putting Descartes before des horse? , 2003, Environmental Conservation.

[40]  Elizabeth R. Selig,et al.  A Global Analysis of the Effectiveness of Marine Protected Areas in Preventing Coral Loss , 2010, PloS one.

[41]  Benjamin S. Halpern,et al.  Marine reserves have rapid and lasting effects , 2002 .

[42]  Paul J. Ferraro,et al.  Evaluation of biodiversity policy instruments: what works and what doesn't? , 2012 .

[43]  Sangeeta Mangubhai,et al.  Papuan Bird's Head Seascape: emerging threats and challenges in the global center of marine biodiversity. , 2012, Marine pollution bulletin.

[44]  Nicholas A. J. Graham,et al.  Recovery trajectories of coral reef fish assemblages within Kenyan marine protected areas , 2005 .

[45]  P J Somerfield,et al.  Marine reserves: fish life history and ecological traits matter. , 2010, Ecological applications : a publication of the Ecological Society of America.

[46]  P. Sanchez‐Jerez,et al.  Multi-scale spatial heterogeneity, habitat structure, and the effect of marine reserves on Western Mediterranean rocky reef fish assemblages , 2004 .

[47]  Gerald R. Allen,et al.  Reef fishes of the Bird’s Head Peninsula, West Papua, Indonesia , 2009 .

[48]  W. Sutherland,et al.  The need for evidence-based conservation. , 2004, Trends in ecology & evolution.

[49]  R. Grosberg,et al.  Climate Change, Human Impacts, and the Resilience of Coral Reefs , 2003, Science.

[50]  Benjamin S. Halpern,et al.  THE IMPACT OF MARINE RESERVES: DO RESERVES WORK AND DOES RESERVE SIZE MATTER? , 2003 .

[51]  Paul J. Ferraro,et al.  Protected areas reduced poverty in Costa Rica and Thailand , 2010, Proceedings of the National Academy of Sciences.

[52]  Subhrendu K. Pattanayak,et al.  Money for Nothing? A Call for Empirical Evaluation of Biodiversity Conservation Investments , 2006, PLoS biology.

[53]  Garry R. Russ,et al.  Density-dependent spillover from a marine reserve: long-term evidence , 2005 .

[54]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[55]  P. Mumby,et al.  Larval dispersal and movement patterns of coral reef fishes, and implications for marine reserve network design , 2015, Biological reviews of the Cambridge Philosophical Society.

[56]  Alexander Pfaff,et al.  Global protected area impacts , 2010, Proceedings of the Royal Society B: Biological Sciences.

[57]  Purwanto,et al.  Cetaceans in the global centre of marine biodiversity , 2014 .

[58]  Paul J. Ferraro,et al.  Counterfactual thinking and impact evaluation in environmental policy , 2009 .

[59]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[60]  Paul R Rosenbaum,et al.  Sensitivity Analysis for m‐Estimates, Tests, and Confidence Intervals in Matched Observational Studies , 2007, Biometrics.

[61]  Robert M Ewers,et al.  Estimates of reserve effectiveness are confounded by leakage. , 2008, Trends in ecology & evolution.

[62]  Naomi Kingston,et al.  Evaluating official marine protected area coverage for Aichi Target 11: appraising the data and methods that define our progress , 2014 .

[63]  M. Spalding,et al.  Reefs at Risk Revisited , 2011 .

[64]  James N. Sanchirico,et al.  Reexamining the science of marine protected areas: linking knowledge to action , 2012 .

[65]  Jasjeet S. Sekhon,et al.  Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching Package for R , 2008 .

[66]  Marco Caliendo,et al.  Some Practical Guidance for the Implementation of Propensity Score Matching , 2005, SSRN Electronic Journal.

[67]  Allan Stewart-Oaten,et al.  ENVIRONMENTAL IMPACT ASSESSMENT: "PSEUDOREPLICATION" IN TIME?' , 1986 .

[68]  Christoph Nolte,et al.  Estimating the impacts of conservation on ecosystem services and poverty by integrating modeling and evaluation , 2015, Proceedings of the National Academy of Sciences.

[69]  P. Rosenbaum.  Design of Observational Studies , 2009, Springer Series in Statistics.

[70]  R. Naidoo,et al.  Impacts of Marine Protected Areas on Fishing Communities , 2010, Conservation biology : the journal of the Society for Conservation Biology.

[71]  R. Prabhakar,et al.  Decentralization for cost-effective conservation , 2009, Proceedings of the National Academy of Sciences.

[72]  Robert L. Pressey,et al.  Poverty and protected areas: An evaluation of a marine integrated conservation and development project in Indonesia , 2014 .

[73]  Hugh P. Possingham,et al.  Efficiency, costs and trade-offs in marine reserve system design , 2005 .

[74]  D. Bellwood,et al.  Confronting the coral reef crisis , 2004, Nature.

[75]  Marc Hockings,et al.  Managers consider multiple lines of evidence important for biodiversity management decisions. , 2012, Journal of environmental management.

[76]  Carrie V. Kappel,et al.  A Global Map of Human Impact on Marine Ecosystems , 2008, Science.

[77]  Paul J. Ferraro,et al.  More strictly protected areas are not necessarily more protective: evidence from Bolivia, Costa Rica, Indonesia, and Thailand , 2013 .