First audit of macroinvertebrate samples from an EU Water Framework Directive monitoring program: human error greatly lowers precision of assessment results

Abstract Invertebrates are often used in biological monitoring of soil and water ecosystems. Because of the huge number of invertebrate species, sample processing (sorting and identification) is a labor-intensive and often difficult task that is prone to error. These errors can bias assessment results, which often are used by environmental managers to guide funding decisions for costly restoration measures. However, quality control of assessment results is not implemented in many freshwater monitoring programs. We conducted the first audit of an official European freshwater monitoring program based on 414 macroinvertebrate samples from streams and rivers in Germany. The samples were collected by personnel at 7 different commercial environmental laboratories using the European Union (EU) Water Framework Directive protocol. We audited 12% of all samples at 3 different levels: 1) a sorting audit, 2) an identification audit, and 3) a total audit based on both sorting and identification. The sorting audit revealed that 29% of the specimens and every 5th taxon (20.6%) had been overlooked by the primary analyst. Differences in sorting were correlated with taxon body size (r  =  0.61, p < 0.001). The identification audit showed that >30% of taxa differed between the results of the primary analysts and auditors. Taxa considered difficult to identify were not more prone to error than were taxa considered easier to identify. Primary analysts and auditors assigned 34% of audited samples to different quality classes in ≥1 of 3 assessment modules (organic pollution, acidification, and general degradation). For 16% of the samples, these changes resulted in a different final ecological assessment. Such a high rate of differences between primary analysts and auditors could lead to ineffective allocation of several million Euros. Our results clearly illustrate the need for adequate quality control and auditing in freshwater monitoring.

[1]  D. Hering,et al.  Assessing streams in Germany with benthic invertebrates: development of a practical standardised protocol for macroinvertebrate sampling and sorting , 2004 .

[2]  S. Pauls,et al.  Taxonomy and systematics: contributions to benthology and J-NABS , 2010, Journal of the North American Benthological Society.

[3]  Christian K. Feld,et al.  Assessing streams in Germany with benthic invertebrates: Development of a multimetric invertebrate based assessment system , 2004 .

[4]  R. Clarke,et al.  Assessing the impact of errors in sorting and identifying macroinvertebrate samples , 2006, Hydrobiologia.

[5]  Peter Haase,et al.  The STAR project: context, objectives and approaches , 2006, Hydrobiologia.

[6]  Mike T. Furse,et al.  A preliminary classification of running‐water sites in Great Britain based on macro‐invertebrate species and the prediction of community type using environmental data , 1984 .

[7]  P. Haase,et al.  Testing different sorting techniques in macroinvertebrate samples from running waters , 2004 .

[8]  Daniel Chessel,et al.  A fuzzy coding approach for the analysis of long‐term ecological data , 1994 .

[9]  R. Clarke,et al.  Effects of locally rare taxa on the precision and sensitivity of RIVPACS bioassessment of freshwaters , 2006 .

[10]  J. Edwards,et al.  The Global Biodiversity Information Facility (GBIF) , 2007 .

[11]  Donald Edward,et al.  AusRivAS: using macroinvertebrates to assess ecological condition of rivers in Western Australia , 1999 .

[12]  J. Wright,et al.  Quality assurance and RIVPACS. , 2000 .

[13]  James R. Karr,et al.  Sustaining living rivers , 2000, Hydrobiologia.

[14]  Eduardo Couto Dalcin,et al.  Data quality concepts and techniques applied to taxonomic databases , 2005 .

[15]  James B. Stribling,et al.  Determining the quality of taxonomic data , 2003, Journal of the North American Benthological Society.

[16]  Barbara J. Downes,et al.  Spatial variation in the distribution of stream invertebrates: implications of patchiness for models of community organization , 1993 .

[17]  David P. Larsen,et al.  Variability in stream macroinvertebrates at multiple spatial scales , 2001 .

[18]  U. Braukmann,et al.  Conceptual study — An improved method to assess acidification in German streams by using benthic macroinvertebrates , 2004 .

[19]  P. Usseglio-Polatera.  Représentation graphique synthétique de la signification écologique d'un peuplement : application aux macroinvertébrés du Rhône à Lyon , 1991 .

[20]  Richard K. Johnson,et al.  Assessing acid stress in Swedish boreal and alpine streams using benthic macroinvertebrates , 2004, Hydrobiologia.

[21]  R. Clarke,et al.  Errors and uncertainty in bioassessment methods – major results and conclusions from the STAR project and their application using STARBUGS , 2006, Hydrobiologia.

[22]  P. Verdonschot,et al.  Towards a multimetric index for the assessment of Dutch streams using benthic macroinvertebrates , 2004, Hydrobiologia.

[23]  P. Haase,et al.  Effects of sampling microhabitats with low coverage within the STAR/AQEM macroinvertebrate sampling protocol on stream assessment , 2008 .

[24]  J. Webster,et al.  Patch Dynamics in Lotic Systems: The Stream as a Mosaic , 1988, Journal of the North American Benthological Society.

[25]  C. Townsend,et al.  Patch history, invertebrate patch dynamics and heterogeneous community composition : perspectives from a manipulative stream experiment , 2007 .

[26]  Jeffrey D. Ostermiller,et al.  Effects of sampling error on bioassessments of stream ecosystems: application to RIVPACS-type models , 2004, Journal of the North American Benthological Society.

[27]  Susan J. Nichols,et al.  Sample Variability Influences on the Precision of Predictive Bioassessment , 2006, Hydrobiologia.

[28]  Richard K. Johnson,et al.  Assessing acid stress in Swedish boreal and alpine streams using benthic macroinvertebrates , 2004, Hydrobiologia.

[29]  Colin R. Townsend,et al.  The Patch Dynamics Concept of Stream Community Ecology , 1989, Journal of the North American Benthological Society.

[30]  J. Stribling,et al.  Data quality, performance, and uncertainty in taxonomic identification for biological assessments , 2008, Journal of the North American Benthological Society.

[31]  R. Clarke,et al.  Within-stream variability of benthic invertebrate samples and EU Water Framework Directive assessment results , 2008 .

[32]  Søren E. Larsen,et al.  Comparison of macroinvertebrate sampling methods in Europe , 2006, Hydrobiologia.

[33]  R. Clarke,et al.  Effects of sampling and sub-sampling variation using the STAR-AQEM sampling protocol on the precision of macroinvertebrate metrics , 2006, Hydrobiologia.

[34]  C. Hawkins,et al.  Measuring and controlling data quality in biological assemblage surveys with special reference to stream benthic macroinvertebrates , 2003 .

[35]  S. Weisberg,et al.  Variability in the Identification and Enumeration of Marine Benthic Invertebrate Samples and Its Effect on Benthic Assessment Measures , 2003, Environmental monitoring and assessment.

[36]  Ralph T. Clarke,et al.  Errors and uncertainty in bioassessment methods : major results and conclusions from the STAR project and their application using STARBUGS , 2006 .