Evaluating data quality collected by volunteers for first-level inspection of hydraulic structures in mountain catchments

Volunteers have been trained to perform first-level inspections of hydraulic structures within campaigns promoted by civil protection of Friuli Venezia Giulia (Italy). Two inspection forms and a learning session were prepared to standardize data collection on the functional status of bridges and check dams. In all, 11 technicians and 25 volunteers inspected a maximum of six structures in Pontebba, a mountain community within the Fella Basin. Volunteers included civil-protection volunteers, geosciences and social sciences students. Some participants carried out the inspection without attending the learning session. Thus, we used the mode of technicians in the learning group to distinguish accuracy levels between volunteers and technicians. Data quality was assessed by their accuracy, precision and completeness. We assigned ordinal scores to the rating scales in order to get an indication of the structure status. We also considered performance and feedback of participants to identify corrective actions in survey procedures. Results showed that volunteers could perform comparably to technicians, but only with a given range in precision. However, a completeness ratio (question / parameter) was still needed any time volunteers used unspecified options. Then, volunteers’ ratings could be considered as preliminary assessments without replacing other procedures. Future research should consider advantages of mobile applications for data-collection methods.

[1]  Krista G. Hilchey,et al.  A review of citizen science and community-based environmental monitoring: issues and opportunities , 2011, Environmental monitoring and assessment.

[2]  E Nicholson,et al.  Community data--where does the value lie? Assessing confidence limits of community collected water quality data. , 2002, Water science and technology : a journal of the International Association on Water Pollution Research.

[3]  Alice Brandon,et al.  Can volunteers provide reliable data for forest vegetation surveys , 2003 .

[4]  Cristina Gouveia,et al.  New approaches to environmental monitoring: the use of ICT to explore volunteered geographic information , 2008 .

[5]  Marco Borga,et al.  Hydrometeorological Analysis of the 29 August 2003 Flash Flood in the Eastern Italian Alps , 2007 .

[6]  C. Potter,et al.  Citizen science as seen by scientists: Methodological, epistemological and ethical dimensions , 2014, Public understanding of science.

[7]  Leska S. Fore,et al.  Assessing the performance of volunteers in monitoring streams. , 2001 .

[8]  Kevin Crowston,et al.  The future of citizen science: emerging technologies and shifting paradigms , 2012, Frontiers in Ecology and the Environment.

[9]  B. Savan,et al.  Volunteer Environmental Monitoring and the Role of the Universities: The Case of Citizens' Environment Watch , 2003, Environmental management.

[10]  F. Cherqui,et al.  The consistency of visual sewer inspection data , 2013 .

[11]  T. Snäll,et al.  Evaluating citizen-based presence data for bird monitoring , 2011 .

[12]  Gloria Bordogna,et al.  A linguistic decision making approach to assess the quality of volunteer geographic information for citizen science , 2014, Inf. Sci..

[13]  Thomas J. Stohlgren,et al.  Improving and integrating data on invasive species collected by citizen scientists , 2010, Biological Invasions.

[14]  M. Jakob,et al.  Debris-flow Hazards and Related Phenomena , 2005 .

[15]  Jan Seibert,et al.  Sensing with boots and trousers qualitative field observations of shallow soil moisture patterns , 2012 .

[16]  Tracy Depue,et al.  The Community Collaborative Rain, Hail, and Snow Network Informal Education for Scientists and Citizens , 2005 .

[17]  R Bjorkland,et al.  A Stream Visual Assessment Protocol (SVAP) for Riparian Landowners , 2001, Environmental monitoring and assessment.

[18]  Melinda Laituri,et al.  User-friendly web mapping: lessons from a citizen science website , 2010, Int. J. Geogr. Inf. Sci..

[19]  Luca Zini,et al.  Debris Flow Phenomena: A Short Overview? , 2012 .

[20]  António Câmara,et al.  Promoting the use of environmental data collected by concerned citizens through information and communication technologies. , 2004, Journal of environmental management.

[21]  R. Whittaker,et al.  Beyond scarcity: citizen science programmes as useful tools for conservation biogeography , 2010 .

[22]  R. Jordan,et al.  Evaluating the Performance of Volunteers in Mapping Invasive Plants in Public Conservation Lands , 2012, Environmental Management.

[23]  Andreas Paul Zischg,et al.  Modelling woody material transport and deposition in alpine rivers , 2011 .

[24]  M. Holub,et al.  Local protection against mountain hazards – state of the art and future needs , 2008 .

[25]  Martha C. Monroe,et al.  Lessons Learned from Evaluations of Citizen Science Programs , 2012, EDIS.

[26]  Christopher J. Seeger,et al.  The role of facilitated volunteered geographic information in the landscape planning and site design process , 2008 .

[27]  Alexandre Remaître,et al.  Morphology and sedimentology of a complex debris flow in a clay‐shale basin , 2005 .

[28]  Kenneth T. Yetman,et al.  USING MARYLAND'S STREAM CORRIDOR ASSESSMENT SURVEY TO PRIORITIZE WATERSHED RESTORATION EFFORTS 1 , 2002 .

[29]  J. Enders,et al.  Measuring Community Awareness and Preparedness for Emergencies , 2001 .

[30]  R. Bonney,et al.  Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy , 2009 .

[31]  Helen E. Roy,et al.  Guide to citizen science: developing, implementing and evaluating citizen science to study biodiversity and theenvironment in the UK , 2012 .

[32]  M. Goodchild Citizens as sensors: the world of volunteered geography , 2007 .

[33]  Miriam J. Metzger,et al.  The credibility of volunteered geographic information , 2008 .

[34]  Mohammad Najafi,et al.  Development of a culvert inventory and inspection framework for asset management of road structures , 2011 .

[35]  J. Reese Voshell,et al.  Volunteer Biological Monitoring: Can It Accurately Assess the Ecological Condition of Streams? , 2002 .

[36]  Neil D. Burgess,et al.  Monitoring Matters: Examining the Potential of Locally-based Approaches , 2005, Biodiversity & Conservation.

[37]  H. Levrel,et al.  Enhancing citizen contributions to biodiversity science and public policy , 2008 .

[38]  A. W. Galloway,et al.  The Reliability of Citizen Science: A Case Study of Oregon White Oak Stand Surveys , 2006 .

[39]  C. de Jong,et al.  Linking ICT and society in early warning and adaptation to hydrological extremes in mountains , 2013 .

[40]  Suzanne Lacasse,et al.  A conceptual framework for quantitative estimation of physical vulnerability to landslides , 2008 .

[41]  Nicola Berni,et al.  Ex post damage assessment: an Italian experience , 2014 .

[42]  Michael F. Goodchild,et al.  Assuring the quality of volunteered geographic information , 2012 .

[43]  Nick Reid,et al.  Can Volunteers Collect Data that are Comparable to Professional Scientists? A Study of Variables Used in Monitoring the Outcomes of Ecosystem Rehabilitation , 2012, Environmental Management.

[44]  Markus Holub,et al.  Mitigating mountain hazards in Austria – legislation, risk transfer, and awareness building , 2009 .