Synthesizing and standardizing criteria for the evaluation of sustainability indicators in the water sector

Indicators are one of the tools available in planning and management projects that aid in the decision-making process and the monitoring of those decisions on the path to sustainable use and management of water and natural resources. However, the quality and trustworthiness of the indicators depend on the constant improvement in the means to assess and design criteria sets. The identification of criteria to evaluate indicators and its subsequence selection are not an ordinary task. The research identified a proliferation of unconsolidated criteria in use in the sustainability and water resource management domains. In response, a process of synthesis and consolidation was undertaken in order to reduce the level of redundancies and to identify possible candidates for “core criteria” that are identified as being a relevant part of most evaluation frameworks. A representative collection of sources from the specialized literature was screened for evaluation criteria. Altogether, 74 sources were assessed, comprising 346 mentions of criteria applied for indicator assessment. A detailed synthesis was performed to organize the criteria and identify possible redundancies. The analysis allowed a reduction from the 346 initial criteria to 60 unique criteria. The study offers a standard title and description for each criterion, contributing to improve clarity and avoid ambiguity. The criteria were also ranked to identify which criteria were in more systemic use. Of the 60 criteria found, the 12 most cited were identified as possible core criteria for framework development. Also, in order to facilitate the design of indicator sets, all 60 criteria were divided into two approaches (scientific/top-down or end-use/bottom-up). This study identified significant redundancies and a lack of standardization in the use of criteria, and it also ranked criteria to facilitate multi-method framework development. Thus, it is essential that indicator designers not only consider criteria that have some level of standardization to be able to compare and communicate with other agencies and communities but also consider how to utilize core criteria in the design of indicator sets.