The Choice of the Sustainability Assessment Tool Matters: Differences in Thematic Scope and Assessment Results

While the number of farm-level sustainability assessment tools is growing rapidly, concerns are raised on whether the assessment results of different tools present similar and valid conclusions about the sustainability performance of farms. In this paper we analysed the thematic scope of sustainability assessment tools, and compared assessment results from sustainability assessment tools. A coverage analysis of four tools (RISE, SAFA, PG and IDEA) demonstrated the diversity in approaches to assess sustainability at farm level. Tool developers select different (sub)themes and indicators, and apply different methods for measurement and aggregation of scores. This variability in approaches results not only in different tools, but can also result in different conclusions on the sustainability performance of farms. Decisions made in the development of a sustainability assessment tool need to be transparent to understand and explain the results of a tool and support farmers in a sustainable development of their farm. To improve the transparency in sustainability assessment tools we presented a framework describing decisions made in the development of a tool. An increased transparency in sustainability assessment tools can reduce the risk on distorted assessment results and actions, and contribute to the trust and relevance of future sustainability assessments.

[1]  M B Galan,et al.  ISO 14 001 at the farm level: analysis of five methods for evaluating the environmental impact of agricultural practices. , 2007, Journal of environmental management.

[2]  Biagio Ciuffo,et al.  A systemic framework for sustainability assessment , 2015 .

[3]  Alexandros Gasparatos,et al.  Embedded value systems in sustainability assessment tools and their implications. , 2010, Journal of environmental management.

[4]  L. Green,et al.  Animal-Based Measures for the Assessment of Welfare State of Dairy Cattle, Pigs and Laying Hens: Consensus of Expert Opinion , 2003, Animal Welfare.

[5]  Catriona J. MacLeod,et al.  When experts disagree: the need to rethink indicator selection for assessing sustainability of agriculture , 2017, Environment, Development and Sustainability.

[6]  Hugo Fjelsted Alrøe,et al.  Insight, part of a Special Feature on Multicriteria Assessment of Food System Sustainability Observing the observers: uncovering the role of values in research assessments of organic food systems , 2014 .

[7]  Didier Stilmant,et al.  Sustainability indicators for livestock farming. A review , 2012, Agronomy for Sustainable Development.

[8]  Stephen Morse,et al.  Breaking through the Glass Ceiling: Who really cares about sustainability indicators? , 2001 .

[9]  D. Makowski,et al.  Agri-environmental indicators to assess cropping and farming systems. A review , 2011, Agronomy for Sustainable Development.

[10]  J. Martínez-Alier,et al.  Weak Comparability of Values as a Foundation for Ecological Economics , 1998 .

[11]  S. Bell,et al.  Sustainability Indicators: Measuring the Immeasurable? , 1999 .

[12]  Mark Reed,et al.  An adaptive learning process for developing and applying sustainability indicators with local communities , 2006 .

[13]  Ruud B.M. Huirne,et al.  Development and application of a multi-attribute sustainability function for Dutch dairy farming systems , 2006 .

[14]  M Andreoli,et al.  Farm sustainability evaluation: methodology and practice , 2000 .

[15]  A. Scolobig,et al.  Choosing the most appropriate sustainability assessment tool , 2012 .

[16]  Víctor-Andrés Cloquell-Ballester,et al.  Indicators validation for the improvement of environmental and social impact quantitative assessment , 2006 .

[17]  N. Schäpke,et al.  Action, research and participation: roles of researchers in sustainability transitions , 2014, Sustainability Science.

[18]  Susanne Padel,et al.  Insight, part of a Special Feature on Multicriteria Assessment of Food System Sustainability Key characteristics for tool choice in indicator-based sustainability assessment at farm level , 2014 .

[19]  Claus G. Sørensen,et al.  Assessing sustainability at farm-level : Lessons learned from a comparison of tools in practice , 2016 .

[20]  Paul B. Thompson,et al.  The varieties of sustainability , 1992 .

[21]  Noelle Aarts,et al.  Dealing with Ambivalence: Farmers' and Consumers' Perceptions of Animal Welfare in Livestock Breeding , 2002 .

[22]  James Hansen,et al.  Is agricultural sustainability a useful concept , 1996 .

[23]  Sharachchandra Lele,et al.  Sustainability and the Scientist's Burden , 1996 .

[24]  M. Vanclooster,et al.  SAFE - A hierarchical framework for assessing the sustainability of agricultural systems , 2007 .

[25]  Christian Bockstaller,et al.  How to validate environmental indicators , 2003 .

[26]  Christian Schader,et al.  Research, part of a Special Feature on Multicriteria Assessment of Food System Sustainability Scope and precision of sustainability assessment approaches to food systems , 2014 .

[27]  Claudia R. Binder,et al.  Considering the normative, systemic and procedural dimensions in indicator-based sustainability assessments in agriculture , 2010 .

[28]  Fritz Hani,et al.  RISE, a Tool for Holistic Sustainability Assessment at the Farm Level , 2003 .

[29]  Claus G. Sørensen,et al.  Generic sustainability assessment themes and the role of context: the case of Danish maize for German biogas. , 2015 .

[30]  Rudolf de Groot,et al.  A conceptual framework for selecting environmental indicator sets , 2008 .

[31]  Ludwig Lauwers,et al.  Research, part of a Special Feature on Multicriteria Assessment of Food System Sustainability Reflection on the development process of a sustainability assessment tool: learning from a Flemish case , 2014 .

[32]  Laurence Guichard,et al.  Comparison of methods to assess the sustainability of agricultural systems. A review , 2011, Agronomy for Sustainable Development.

[33]  Frank Nevens,et al.  MOTIFS: a monitoring tool for integrated farm sustainability , 2008, Agronomy for Sustainable Development.

[34]  Claus G. Sørensen,et al.  Assessing the Sustainability Performance of Organic Farms in Denmark , 2016 .

[35]  S. Padel,et al.  Public goods and farming , 2012 .

[36]  Christian Bockstaller,et al.  Sustainability criteria Issues in evaluating sustainability of farming systems with indicators , 2015 .

[37]  Giuseppe Munda,et al.  Cost-benefit analysis in integrated environmental assessment: some methodological issues , 1996 .

[38]  P. Girardin,et al.  Assessing farm sustainability with the IDEA method – from the concept of agriculture sustainability to case studies on farms , 2008 .

[39]  I. D. Boer,et al.  Evaluation of indicators to assess the environmental impact of dairy production systems , 2005 .