The paradox of expert judgment in rivers ecological monitoring.

A great investment has been done in the last decades in the development of numerical and qualitative assessment methods to classify the ecological quality of water bodies. Yet, in spite of all attempts to avoid subjectivity, expert judgment is still used at numerous steps of the ecological classification and is considered by some authors as indispensible for management purposes. Thus, the aim of this study is to test the hypothesis that expert judgment, when done by the adequate experts (limnologists/river ecologist) with experience in the study area (i.e., natural conditions and expected communities), could be as good as quantitative indices and measures (i.e., result in the same classification), but quicker and with lower cost. For that we compared the classifications (on 13 aspects of rivers ecosystems) attributed by two experts to 20 sites (10 each) located in their study areas, with the classifications of ecological quality based on biological indices (for invertebrates and diatoms), hydromorphology and water chemistry, calculated by an independent team. Our results show that assessments made by experts and those calculated through indices (biological quality and hydromorphology) are globally very similar (RELATE test; Rho = 0.442; p < 0.001, 999 permutations). Most differences were of one class and experts tended to attribute a better condition than indices to the best quality sites but a worse condition to the worse quality sites. A Principal Components Analysis revealed that sites to which experts attributed a moderate quality had higher nitrate concentration and pH but were well oxygenated. The sites classified as poor and bad where those with stronger modifications in their habitats (given by the higher values of HMS). The difference between experts and indices is small but still represents 15% of sites, and includes both situations: the experts or the indices lead to the need of measures (i.e., classifications below class Good). Experts' evaluations on hydromorphological conditions of the channel and margins are also significantly correlated with the quality assessments made by the field team that has no experience in the study area (Rh0 = 0.518; p = 0.001; 999 permutation), indicating geographic independence in the expert judgment. We concluded that expert judgment could be used in the determination of streams and rivers ecological quality, saving money and time and helping to redirect monitoring funds to actual implementation of restoration measures. Yet, classification' scoring methods may still be useful for a better targeting of restoration measures.

[1]  Joaquín Izquierdo,et al.  Balancing consistency and expert judgment in AHP , 2011, Math. Comput. Model..

[2]  John R. Olson,et al.  The reference condition: predicting benchmarks for ecological and water-quality assessments , 2010, Journal of the North American Benthological Society.

[3]  Michele Scardi,et al.  An expert system based on fish assemblages for evaluating the ecological quality of streams and rivers , 2008, Ecol. Informatics.

[4]  G. Scrimgeour,et al.  Aquatic Ecosystem Health and Integrity: Problems and Potential Solutions , 1996, Journal of the North American Benthological Society.

[5]  S Birk,et al.  Intercalibrating classifications of ecological status: Europe's quest for common management objectives for aquatic ecosystems. , 2013, The Science of the total environment.

[6]  D. Goffaux,et al.  Assessing river biotic condition at a continental scale: a European approach using functional metrics and fish assemblages , 2006 .

[7]  James R. Karr,et al.  Ecological perspective on water quality goals , 1981 .

[8]  J. Soininen,et al.  Comparative study of monitoring South-Finnish rivers and streams using macroinvertebrate and benthic diatom community structure , 2004, Aquatic Ecology.

[9]  E. Papastergiadou,et al.  Comparability of river quality assessment using macrophytes: a multi-step procedure to overcome biogeographical differences. , 2014, The Science of the total environment.

[10]  B. Statzner,et al.  Responses of freshwater biota to human disturbances: contribution of J-NABS to developments in ecological integrity assessments , 2010, Journal of the North American Benthological Society.

[11]  S. Sabater,et al.  Water quality assessment of rivers using diatom metrics across Mediterranean Europe: a methods intercalibration exercise. , 2014, The Science of the total environment.

[12]  Martin Kernan,et al.  Assessing the ecological status in the context of the European Water Framework Directive: where do we go now? , 2014, The Science of the total environment.

[13]  Steven Degraer,et al.  Assessing coastal benthic macrofauna community condition using best professional judgement--developing consensus across North America and Europe. , 2010, Marine pollution bulletin.

[14]  Maria João Feio,et al.  Predictive Models for Freshwater Biological Assessment: Statistical Approaches, Biological Elements and the Iberian Peninsula Experience: A Review , 2011 .

[15]  Mike Acreman,et al.  Defining environmental river flow requirements ? a review , 2004 .

[16]  DIATMOD: diatom predictive model for quality assessment of Portuguese running waters , 2012, Hydrobiologia.

[17]  António de Jesus Fernandes de Matos,et al.  Ordenamento do território e desenvolvimento regional , 2000 .

[18]  Maria Teresa Ferreira,et al.  Estimating species tolerance to human perturbation: Expert judgment versus empirical approaches , 2011 .

[19]  Andrea Buffagni,et al.  The European reference condition concept: A scientific and technical approach to identify minimally-impacted river ecosystems. , 2012, The Science of the total environment.

[20]  K. R. Clarke,et al.  Change in marine communities : an approach to statistical analysis and interpretation , 2001 .

[21]  Ángel Borja,et al.  Assessing the environmental quality status in estuarine and coastal systems: Comparing methodologies and indices , 2008 .

[22]  N. Prat,et al.  Comparability of ecological quality boundaries in the Mediterranean basin using freshwater benthic invertebrates. Statistical options and implications. , 2014, The Science of the total environment.

[23]  M. T. Ferreira,et al.  Choosing the best method for stream bioassessment using macrophyte communities: Indices and predictive models , 2011 .

[24]  Y. Tupinier Henri Tachet, Philippe Richoux, Michel Bournaud, Philippe Usseglio-Polatera - Invertébrés d'eau douce : systématique, biologie, écologie. 2000 , 2002 .

[25]  Colin Camerer,et al.  The process-performance paradox in expert judgment - How can experts know so much and predict so badly? , 1991 .

[26]  M. Feio,et al.  Diatoms and macroinvertebrates provide consistent and complementary information on environmental quality , 2007 .

[27]  Igor Linkov,et al.  Multi-criteria decision analysis to select metrics for design and monitoring of sustainable ecosystem restorations , 2013 .

[28]  Anne Lyche Solheim,et al.  Lake Ecological Assessment Systems and Intercalibration for the European Water Framework Directive: Aims, Achievements and Further Challenges , 2011 .

[29]  S. Weisberg,et al.  The level of agreement among experts applying best professional judgment to assess the condition of benthic infaunal communities , 2008 .

[30]  Carolina Solà,et al.  Indice de Macrófitos Fluviales (IMF), una nueva herramienta para evaluar el estado ecológico de los ríos mediterráneos , 2015 .

[31]  Juan Bald,et al.  Using historical data, expert judgement and multivariate analysis in assessing reference conditions and benthic ecological status, according to the European Water Framework Directive. , 2007, Marine pollution bulletin.

[32]  S. Dufour,et al.  From the myth of a lost paradise to targeted river restoration: forget natural references and focus on human benefits , 2009 .

[33]  Mark A. Burgman,et al.  Expert Status and Performance , 2011, PloS one.

[34]  António N. Pinheiro,et al.  BENCHMARKING RIVER HABITAT IMPROVEMENT , 2012 .

[35]  Kevin B. Lunde,et al.  Evaluating the adequacy of a reference-site pool for ecological assessments in environmentally complex regions , 2016, Freshwater Science.

[36]  John L Stoddard,et al.  Setting expectations for the ecological condition of streams: the concept of reference condition. , 2005, Ecological applications : a publication of the Ecological Society of America.

[37]  J Ferreira,et al.  Least Disturbed Condition for European Mediterranean rivers. , 2014, The Science of the total environment.

[38]  N. Willby,et al.  Harmonising the bioassessment of large rivers in the absence of near‐natural reference conditions – a case study of the Danube River , 2012 .

[39]  Anne Courrat,et al.  Three hundred ways to assess Europe's surface waters: An almost complete overview of biological methods to implement the Water Framework Directive , 2012 .