Deconstructing criteria and assessment tools to build agri-sustainability indicators and support farmers' decision-making process

Abstract In the review of academic literature, numerous papers present either a list of indicator criteria or partially revised sustainability assessment tools of agri-food systems. However, neither a complete analysis and discussion about the criteria utilised by evaluators nor a compared examination and subsequent frameworks categorization have been fully developed by researchers. This study aims to fill this twofold gap by investigating the main issues related to the choice of a tool for the sustainability assessment of an agri-system. This task is conducted in three steps: firstly, we analyse the criteria an indicator should match to be included in an evaluation; secondly, we categorise 15 of the most important agriculture sustainability frameworks to discuss effectiveness in evaluating sustainability for each category, finally, we compare the categories and emphasise differences to highlight the possible application of each framework and hence guide the practitioner in the framework selection process. Our analysis identifies the complementarity between bottom-up and top-down approach and the impossibility of identifying a priori the best framework, although a combination of both approaches could prove to be a valuable, alternative option.

[1]  E. Galdeano-Gómez,et al.  Exploring Synergies Among Agricultural Sustainability Dimensions: An Empirical Study on Farming System in Almería (Southeast Spain) , 2017 .

[2]  O. Masera Sustentabilidad y manejo de recursos naturalesel marco de evaluación MESMIS , 1999 .

[3]  Union européenne,et al.  Handbook on Constructing Composite Indicators: Methodology and User Guide , 2008 .

[4]  John Tzilivakis,et al.  The development and use of farm level indicators in England , 2004 .

[5]  P. Moreno‐Casasola,et al.  Environmental and socio-economic sustainability of chinampas (raised beds) in Xochimilco, Mexico City , 2013 .

[6]  Fritz Hani,et al.  RISE, a Tool for Holistic Sustainability Assessment at the Farm Level , 2003 .

[7]  Anne Vanasse,et al.  Development of agri-environmental indicators to assess dairy farm sustainability in Quebec, Eastern Canada , 2012 .

[8]  Walter A.H. Rossing,et al.  Integrative modelling approaches for analysis of impact of multifunctional agriculture: A review for France, Germany and The Netherlands , 2007 .

[9]  T. Dantsis,et al.  A methodological approach to assess and compare the sustainability level of agricultural plant production systems , 2010 .

[10]  Didier Stilmant,et al.  Sustainability indicators for livestock farming. A review , 2012, Agronomy for Sustainable Development.

[11]  D. Rigby,et al.  Constructing a farm level indicator of sustainable agricultural practice , 2001 .

[12]  Arnim Wiek,et al.  Solution spaces for decision-making—a sustainability assessment tool for city-regions , 2005 .

[13]  Rudolf de Groot,et al.  A conceptual framework for selecting environmental indicator sets , 2008 .

[14]  C. Binder,et al.  Trade‐off analysis for agro‐ecological indicators: application of Sustainable Solution Space to maize cropping systems in northern Italy , 2010 .

[15]  Susanne Padel,et al.  Insight, part of a Special Feature on Multicriteria Assessment of Food System Sustainability Key characteristics for tool choice in indicator-based sustainability assessment at farm level , 2014 .

[16]  Xu Cheng,et al.  Analysis and Design of Agricultural Sustainability Indicators System , 2007 .

[17]  R. Cassen Our common future: report of the World Commission on Environment and Development , 1987 .

[18]  Ruud B.M. Huirne,et al.  Modelling worker physical health and societal sustainability at farm level: an application to conventional and organic dairy farming , 2007 .

[19]  Jason Potts,et al.  Seeking Sustainability: COSA Preliminary Analysis of Sustainability Initiatives in the Coffee Sector , 2008 .

[20]  J. A. Gómez-Limón,et al.  Empirical evaluation of agricultural sustainability using composite indicators. , 2010 .

[21]  Philip Woodhouse,et al.  Family farm sustainability in southern Brazil: An application of agri-environmental indicators , 2008 .

[22]  M. Vanclooster,et al.  SAFE - A hierarchical framework for assessing the sustainability of agricultural systems , 2007 .

[23]  Henk Udo,et al.  ASSESSMENT OF THE CONTRIBUTION OF SUSTAINABILITY INDICATORS TO SUSTAINABLE DEVELOPMENT: A NOVEL APPROACH USING FUZZY SET THEORY , 2001 .

[24]  F. Sottile,et al.  A methodology for the sustainability assessment of agri-food systems: an application to the Slow Food Presidia project. , 2014 .

[25]  Luca Bechini,et al.  Integrated sustainability assessment of cropping systems with agro-ecological and economic indicators in northern Italy , 2010 .

[26]  J. A. Gómez-Limón,et al.  Sustainability assessment of olive groves in Andalusia: A methodological proposal , 2010 .

[27]  Stefan Seuring,et al.  From a literature review to a conceptual framework for sustainable supply chain management , 2008 .

[28]  Martin Kaltschmitt,et al.  Sustainable Crop Production: Definition and Methodological Approach for Assessing and Implementing Sustainability , 1999 .

[29]  Claudia R. Binder,et al.  Considering the normative, systemic and procedural dimensions in indicator-based sustainability assessments in agriculture , 2010 .

[30]  P. Lescoat,et al.  AVIBIO: a method to assess the sustainability of the organic poultry industry. , 2012 .

[31]  M. Meul,et al.  Learning through stakeholder involvement in the implementation of MOTIFS: an integrated assessment model for sustainable farming in Flanders , 2011 .

[32]  A. Smyth,et al.  A framework for evaluating sustainable land management , 1995 .

[33]  Dan Rigby,et al.  The Sustainability of Agricultural Systems , 1997 .

[34]  S. Padel,et al.  Public goods and farming , 2012 .

[35]  Christian Schader,et al.  Research, part of a Special Feature on Multicriteria Assessment of Food System Sustainability Scope and precision of sustainability assessment approaches to food systems , 2014 .

[36]  Stephen Morse,et al.  Sustainability indicators: the problem of integration , 2001 .

[37]  Marnik Vanclooster,et al.  Framework for assessing sustainability levels in Belgian agricultural systems - SAFE. Part 1: Sustainable production and consumption patterns. Final report - SPSD II CP 28 , 2006 .

[38]  A. Smyth,et al.  FELSM: An international framework for evaluating sustainable land management , 1993 .

[39]  Claudia R. Binder,et al.  Sustainability solution space of the Swiss milk value added chain , 2012 .

[40]  Brian R. Keeble BSc Mbbs Mrcgp The Brundtland report: ‘Our common future’ , 1988 .

[41]  H. Keulen,et al.  Multiscale Methodological Framework to Derive Criteria and Indicators for Sustainability Evaluation of Peasant Natural Resource Management Systems , 2005 .

[42]  C. Pahl-Wostl,et al.  Research, part of a Special Feature on A Framework for Analyzing, Comparing, and Diagnosing Social-Ecological Systems Comparison of Frameworks for Analyzing Social-ecological Systems , 2013 .

[43]  Shengkui Cheng,et al.  Three dimensions of sustainability of farming practices in the North China Plain A case study from Ningjin County of Shandong Province, PR China , 2005 .

[44]  O. Masera,et al.  Ten years of sustainability evaluation using the MESMIS framework: Lessons learned from its application in 28 Latin American case studies , 2007 .

[45]  E. Schmid,et al.  Stakeholder engagement in prioritizing sustainability assessment themes for smallholder coffee production in Uganda , 2016, Renewable Agriculture and Food Systems.

[46]  Ludwig Lauwers,et al.  A framework for guiding sustainability assessment and on-farm strategic decision making , 2016 .

[47]  E. Lichtfouse Farming for Food and Water Security , 2012, Sustainable Agriculture Reviews.

[48]  Mette Vaarst,et al.  Assessing Sustainability of Smallholder Beef Cattle Farming in Indonesia: A Case Study Using the FAO SAFA Framework , 2016 .

[49]  P. Girardin,et al.  Assessing farm sustainability with the IDEA method – from the concept of agriculture sustainability to case studies on farms , 2008 .

[50]  Claus G. Sørensen,et al.  Assessing the Sustainability Performance of Organic Farms in Denmark , 2016 .

[51]  Marie Thiollet-Scholtus,et al.  Using indicators to assess the environmental impacts of wine growing activity: The INDIGO® method , 2015 .

[52]  Frank Nevens,et al.  MOTIFS: a monitoring tool for integrated farm sustainability , 2008, Agronomy for Sustainable Development.

[53]  M. Fleur,et al.  A comparison of complex expert-based assessment versus quickscan assessment. , 2012 .

[54]  A. Gupta,et al.  Biophysical, chemical and socio-economic indicators for assessing agricultural sustainability in the Chinese coastal zone , 2001 .

[55]  James Hansen,et al.  Is agricultural sustainability a useful concept , 1996 .

[56]  Virginia H. Dale,et al.  Challenges in the development and use of ecological indicators , 2001 .

[57]  Ngai Weng Chan,et al.  An assessment of agricultural sustainability indicators in Bangladesh: review and synthesis , 2012, The Environmentalist.

[58]  F. Graef,et al.  Methods to assess farming sustainability in developing countries. A review , 2015, Agronomy for Sustainable Development.

[59]  Ruud B.M. Huirne,et al.  Development and application of a multi-attribute sustainability function for Dutch dairy farming systems , 2006 .

[60]  Timothy O'Riordan,et al.  What does sustainability really mean , 1985 .

[61]  Hartmut Stützel,et al.  A new method for assessing the sustainability of land-use systems (II): Evaluating impact indicators , 2009 .

[62]  Peter Saling,et al.  Assessing the Environmental-Hazard Potential for Life Cycle Assessment, Eco-Efficiency and SEEbalance (8 pp) , 2005 .

[63]  Steven Van Passel,et al.  Multilevel and multi-user sustainability assessment of farming systems , 2012 .

[64]  Charles J. Kibert,et al.  Developing indicators of sustainability: US experience , 1998 .

[65]  Stefano Tarantola,et al.  Handbook on Constructing Composite Indicators: Methodology and User Guide , 2005 .

[66]  Reimund P. Rötter,et al.  TechnoGIN, a tool for exploring and evaluating resource use efficiency of cropping systems in East and Southeast Asia , 2006 .

[67]  David March,et al.  Steps toward a shared governance response for achieving Good Environmental Status in the Mediterranean Sea , 2014 .

[68]  Alexandros Gasparatos,et al.  Embedded value systems in sustainability assessment tools and their implications. , 2010, Journal of environmental management.

[69]  N. A. Glenn,et al.  A framework for the economic evaluation and selection of sustainability indicators in agriculture , 2000 .

[70]  M. El-Haram,et al.  A critical review of reductionist approaches for assessing the progress towards sustainability , 2008 .

[71]  Claus G. Sørensen,et al.  Assessing sustainability at farm-level : Lessons learned from a comparison of tools in practice , 2016 .

[72]  A. Scolobig,et al.  Choosing the most appropriate sustainability assessment tool , 2012 .

[73]  Ludwig Lauwers,et al.  Research, part of a Special Feature on Multicriteria Assessment of Food System Sustainability Reflection on the development process of a sustainability assessment tool: learning from a Flemish case , 2014 .

[74]  Saurabh Gupta,et al.  An overview of sustainability assessment methodologies , 2009 .

[75]  Eddie A.M. Bokkers,et al.  The Choice of the Sustainability Assessment Tool Matters: Differences in Thematic Scope and Assessment Results , 2017 .

[76]  H. Voet,et al.  A protocol for evaluating the sustainability of agri-food production systems—A case study on potato production in peri-urban agriculture in The Netherlands , 2014 .

[77]  Francesco Sottile,et al.  Evaluating the Sustainability in Complex Agri-Food Systems: The SAEMETH Framework , 2015 .

[78]  S. Wirén-Lehr,et al.  Sustainability in agriculture — an evaluation of principal goal-oriented concepts to close the gap between theory and practice , 2001 .

[79]  J. Derrida Writing and Difference , 1967 .

[81]  Jean Petit,et al.  Evaluation of the environmental impact of agriculture at the farm level: a comparison and analysis of 12 indicator-based methods , 2002 .

[82]  Claudia R. Binder,et al.  Normative, Systemic and Procedural Aspects: A Review of Indicator-Based Sustainability Assessments in Agriculture , 2010 .

[83]  Hartmut Stützel,et al.  A new method for assessing the sustainability of land-use systems (I): Identifying the relevant issues , 2009 .

[84]  Omar Masera,et al.  Evaluating the sustainability of complex socio-environmental systems. the MESMIS framework , 2002 .

[85]  O. Masera,et al.  Sustainability indicators, alternative strategies and trade-offs in peasant agroecosystems: analysing 15 case studies from Latin America , 2011 .

[86]  Ruslan Rainis,et al.  Rice farming sustainability assessment in Bangladesh , 2013, Sustainability Science.

[87]  Jiří Hřebíček,et al.  Optimal set of agri-environmental indicators for the agricultural sector of Czech Republic , 2013 .

[88]  N. Andrieu,et al.  Design of sustainability indicators of the production systems in Brazilian semi-arid area by the analysis of biomass flows , 2007 .

[89]  Valérie Bélanger,et al.  DELTA: An Integrated Indicator-Based Self-Assessment Tool for the Evaluation of Dairy Farms Sustainability in Quebec, Canada , 2015 .

[90]  Arnim Wiek,et al.  Sustainability Potential Analysis (SPA) of landfills – a systemic approach: theoretical considerations , 2007 .

[91]  Catriona J. MacLeod,et al.  When experts disagree: the need to rethink indicator selection for assessing sustainability of agriculture , 2017, Environment, Development and Sustainability.