Statistical matching for conservation science

Abstract The awareness of the need for robust impact evaluations in conservation is growing and statistical matching techniques are increasingly being used to assess the impacts of conservation interventions. Used appropriately matching approaches are powerful tools, but they also pose potential pitfalls. We outlined important considerations and best practice when using matching in conservation science. We identified 3 steps in a matching analysis. First, develop a clear theory of change to inform selection of treatment and controls and that accounts for real‐world complexities and potential spillover effects. Second, select the appropriate covariates and matching approach. Third, assess the quality of the matching by carrying out a series of checks. The second and third steps can be repeated and should be finalized before outcomes are explored. Future conservation impact evaluations could be improved by increased planning of evaluations alongside the intervention, better integration of qualitative methods, considering spillover effects at larger spatial scales, and more publication of preanalysis plans. Implementing these improvements will require more serious engagement of conservation scientists, practitioners, and funders to mainstream robust impact evaluations into conservation. We hope this article will improve the quality of evaluations and help direct future research to continue to improve the approaches on offer.

[1]  Jordi Honey-Rosés,et al.  Mainstreaming Impact Evaluation in Nature Conservation , 2016 .

[2]  L. Joppa,et al.  High and Far: Biases in the Location of Protected Areas , 2009, PloS one.

[3]  K. Chomitz,et al.  Effectiveness of Strict vs. Multiple Use Protected Areas in Reducing Tropical Forest Fires: A Global Analysis Using Matching Methods , 2011, PloS one.

[4]  Matias D. Cattaneo,et al.  Econometric Methods for Program Evaluation , 2018, Annual Review of Economics.

[5]  J. Robalino,et al.  Spillovers from Conservation Programs , 2017 .

[6]  Experimental evaluation of the impact of a payment for environmental services program on deforestation , 2019, Conservation Science and Practice.

[7]  Robert M Ewers,et al.  Estimates of reserve effectiveness are confounded by leakage. , 2008, Trends in ecology & evolution.

[8]  G. Paoli,et al.  Examining protected area effectiveness in Sumatra: importance of regulations governing unprotected lands , 2012 .

[9]  D. Rubin BIAS REDUCTION USING MAHALANOBIS METRIC MATCHING , 1978 .

[10]  Elizabeth A Stuart,et al.  Matching methods for causal inference: A review and a look forward. , 2010, Statistical science : a review journal of the Institute of Mathematical Statistics.

[11]  David Collier,et al.  Understanding Process Tracing , 2011, PS: Political Science & Politics.

[12]  Paul J. Ferraro,et al.  Advances in Measuring the Environmental and Social Impacts of Environmental Programs , 2014 .

[13]  J. Schleicher The environmental and social impacts of protected areas and conservation concessions in South America , 2018, Current Opinion in Environmental Sustainability.

[14]  Subhrendu K. Pattanayak,et al.  Estimating the Impacts of Local Policy Innovation: The Synthetic Control Method Applied to Tropical Deforestation , 2015, PloS one.

[15]  Subhrendu K. Pattanayak,et al.  Money for Nothing? A Call for Empirical Evaluation of Biodiversity Conservation Investments , 2006, PLoS biology.

[16]  P. Legendre Spatial Autocorrelation: Trouble or New Paradigm? , 1993 .

[17]  Y. School Do Property Rights Promote Investment But Cause Deforestation ? Quasi-Experimental Evidence from Nicaragua , 2017 .

[18]  Robert Haining,et al.  Spatial Data Analysis: Theory and Practice , 2003 .

[19]  Katharine R. E. Sims,et al.  An upside to globalization: International outmigration drives reforestation in Nepal , 2018, Global Environmental Change.

[20]  Neil D. Burgess,et al.  Deforestation in an African biodiversity hotspot: extent, variation and the effectiveness of protected areas , 2013 .

[21]  E. Milner‐Gulland,et al.  Impacts of Protected Areas on Local Livelihoods in Cambodia , 2014 .

[22]  T. Ricketts,et al.  Evaluating the impacts of protected areas on human well-being across the developing world , 2019, Science Advances.

[23]  James S. Kagan,et al.  Evidence-Based Causal Chains for Linking Health, Development, and Conservation Actions , 2018, Bioscience.

[24]  Elizabeth A Stuart,et al.  Prognostic score-based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research. , 2013, Journal of clinical epidemiology.

[25]  James N. Sanchirico,et al.  Causal inference in coupled human and natural systems , 2018, Proceedings of the National Academy of Sciences.

[26]  Julia P. G. Jones,et al.  Effectiveness of Community Forest Management at reducing deforestation in Madagascar , 2015 .

[27]  R. G. Davies,et al.  Methods to account for spatial autocorrelation in the analysis of species distributional data : a review , 2007 .

[28]  Edwin L. Pynegar,et al.  The effectiveness of Payments for Ecosystem Services at delivering improvements in water quality: lessons for experiments at the landscape scale , 2018, PeerJ.

[29]  Sebastian Lemire,et al.  Making contribution analysis work: A practical framework for handling influencing factors and alternative explanations , 2012 .

[30]  E. Lambin,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:Global land use change, economic globalization, and the looming land scarcity , 2011 .

[31]  D. Rubin Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation , 2001, Health Services and Outcomes Research Methodology.

[32]  A. Rubin Improving the Teaching of Evidence-Based Practice: Introduction to the Special Issue , 2007 .

[33]  International funding agencies: potential leaders of impact evaluation in protected areas? , 2015, Philosophical Transactions of the Royal Society B: Biological Sciences.

[34]  Michael Lechner A Note on the Common Support Problem in Applied Evaluation Studies , 2000 .

[35]  M. Brockhaus,et al.  Enabling factors for establishing REDD+ in a context of weak governance , 2014 .

[36]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[37]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[38]  T. Virtanen,et al.  Contrasting spatial and temporal trends of protected area effectiveness in mitigating deforestation in Madagascar , 2016 .

[39]  Mark Lunt,et al.  Selecting an Appropriate Caliper Can Be Essential for Achieving Good Balance With Propensity Score Matching , 2013, American journal of epidemiology.

[40]  Marco Caliendo,et al.  Some Practical Guidance for the Implementation of Propensity Score Matching , 2005, SSRN Electronic Journal.

[41]  G. King,et al.  Causal Inference without Balance Checking: Coarsened Exact Matching , 2012, Political Analysis.

[42]  Paul R Rosenbaum,et al.  Sensitivity Analysis for m‐Estimates, Tests, and Confidence Intervals in Matched Observational Studies , 2007, Biometrics.

[43]  Valerie Kapos,et al.  Protected Area Effectiveness in Reducing Conversion in a Rapidly Vanishing Ecosystem: The Brazilian Cerrado , 2014 .

[44]  Kelly W. Jones,et al.  Estimating the Counterfactual Impact of Conservation Programs on Land Cover Outcomes: The Role of Matching and Panel Regression Techniques , 2015, PloS one.

[45]  S. Jeanne Horst,et al.  A Brief Guide to Decisions at Each Step of the Propensity Score Matching Process , 2020 .

[46]  W. Sutherland,et al.  The need for evidence-based conservation. , 2004, Trends in ecology & evolution.

[47]  Katharine R. E. Sims,et al.  Reductions in deforestation and poverty from decentralized forest management in Nepal , 2019, Nature Sustainability.

[48]  P. Rosenbaum,et al.  Amplification of Sensitivity Analysis in Matched Observational Studies , 2009, Journal of the American Statistical Association.

[49]  J. Sekhon,et al.  Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies , 2006, Review of Economics and Statistics.

[50]  J. Geldmann,et al.  Exceptional responders in conservation , 2018, Conservation biology : the journal of the Society for Conservation Biology.

[51]  J. Robalino,et al.  Protecting forests, biodiversity, and the climate: predicting policy impact to improve policy choice , 2012 .

[52]  J. Robalino,et al.  Heterogeneous Local Spillovers from Protected Areas in Costa Rica , 2017, Journal of the Association of Environmental and Resource Economists.

[53]  Andrew S. Pullin,et al.  Effectiveness in Conservation Practice: Pointers from Medicine and Public Health , 2001 .

[54]  Katharine R. E. Sims,et al.  Payments for environmental services supported social capital while increasing land management , 2018, Proceedings of the National Academy of Sciences.

[55]  Johan A. Oldekop,et al.  Impact of protected areas on poverty, extreme poverty, and inequality in Nepal , 2018, Conservation Letters.

[56]  Allen Blackman Evaluating forest conservation policies in developing countries using remote sensing data: An introduction and practical guide , 2013 .

[57]  Jonah Busch,et al.  Does deforestation increase malaria prevalence? Evidence from satellite data and health surveys , 2018, World Development.

[58]  C. Peres,et al.  Conservation performance of different conservation governance regimes in the Peruvian Amazon , 2017, Scientific Reports.

[59]  Richard A. Nielsen,et al.  Why Propensity Scores Should Not Be Used for Matching , 2019, Political Analysis.

[60]  Aaron Christ,et al.  Mixed Effects Models and Extensions in Ecology with R , 2009 .

[61]  L. HARKing: Hypothesizing After the Results are Known , 2002 .

[62]  P. Ferraro,et al.  Measuring the effectiveness of protected area networks in reducing deforestation , 2008, Proceedings of the National Academy of Sciences.

[63]  Jianguo Liu,et al.  Spillover effect offsets the conservation effort in the Amazon , 2018, Journal of Geographical Sciences.

[64]  P. Ferraro,et al.  Quantifying causal mechanisms to determine how protected areas affect poverty through changes in ecosystem services and infrastructure , 2014, Proceedings of the National Academy of Sciences.

[65]  G. Canavire-Bacarreza,et al.  Implications of heterogeneous impacts of protected areas on deforestation and poverty , 2015, Philosophical Transactions of the Royal Society B: Biological Sciences.

[66]  Brian A. Nosek,et al.  The preregistration revolution , 2018, Proceedings of the National Academy of Sciences.

[67]  Gary King,et al.  Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference , 2007, Political Analysis.

[68]  Patrick Meyfroidt,et al.  Approaches and terminology for causal analysis in land systems science , 2016 .