Expert, Crowd, Students or Algorithm: who holds the key to deep‐sea imagery ‘big data’ processing?

1.Recent technological development has increased our capacity to study the deep sea and the marine benthic realm, particularly with the development of multidisciplinary seafloor observatories. Since 2006, Ocean Networks Canada cabled observatories, have acquired nearly 65 TB and over 90,000 hours of video data from seafloor cameras and Remotely Operated Vehicles (ROVs). Manual processing of these data is time-consuming and highly labour-intensive, and cannot be comprehensively undertaken by individual researchers. These videos are a crucial source of information for assessing natural variability and ecosystem responses to increasing human activity in the deep sea. 2.We compared the performance of three groups of humans and one computer vision algorithm in counting individuals of the commercially important sablefish (or black cod) Anoplopoma fimbria, in recorded video from a cabled camera platform at 900 m depth in a submarine canyon in the Northeast Pacific. The first group of human observers were untrained volunteers recruited via a crowdsourcing platform and the second were experienced university students, who performed the task for their ichthyology class. Results were validated against counts obtained from a scientific expert. 3.All groups produced relatively accurate results in comparison to the expert and all succeeded in detecting patterns and periodicities in fish abundance data. Trained volunteers displayed the highest accuracy and the algorithm the lowest. 4.As seafloor observatories increase in number around the world, this study demonstrates the value of a hybrid combination of crowdsourcing and computer vision techniques as a tool to help process large volumes of imagery to support basic research and environmental monitoring. Reciprocally, by engaging large numbers of online participants in deep-sea research, this approach can contribute significantly to ocean literacy and informed citizen input to policy development.

[1]  G. Petterson,et al.  Image Analysis Techniques , 2002 .

[2]  Verena Tunnicliffe,et al.  Observations on the effects of sampling on hydrothermal vent habitat and fauna of Axial Seamount, Juan de Fuca Ridge , 1990 .

[3]  Mehrdad Hajibabaei,et al.  Big Data in Ecology , 2013 .

[4]  Thomas J. Stohlgren,et al.  Assessing citizen science data quality: an invasive species case study , 2011 .

[5]  Arthur C. Gentile.  Mining the Deep Sea , 1979 .

[6]  Paolo Menesatti,et al.  A Novel Morphometry-Based Protocol of Automated Video-Image Analysis for Species Recognition and Activity Rhythms Monitoring in Deep-Sea Fauna , 2009, Sensors.

[7]  Alexandra Branzan Albu,et al.  Automatic fish counting system for noisy deep-sea videos , 2014, 2014 Oceans - St. John's.

[8]  David G. Delaney,et al.  Marine invasive species: validation of citizen science and implications for national monitoring networks , 2007, Biological Invasions.

[9]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..

[10]  William Puech,et al.  Robots in Ecology: Welcome to the machine , 2012 .

[11]  Helen E. Roy,et al.  Important role of citizen science in monitoring UK biodiversity , 2012 .

[12]  Lior Shamir,et al.  Combining Human and Machine Learning for Morphological Analysis of Galaxy Images , 2014, ArXiv.

[13]  David B. Roy,et al.  Statistics for citizen science: extracting signals of change from noisy ecological data , 2014 .

[14]  David N. Bonter,et al.  Citizen Science as an Ecological Research Tool: Challenges and Benefits , 2010 .

[15]  G. Tsechpenakis,et al.  Image Analysis Techniques to Accompany a new In Situ Ichthyoplankton Imaging System , 2007, OCEANS 2007 - Europe.

[16]  Eric Brassart,et al.  Colour Image Segmentation Using Homogeneity Method and Data Fusion Techniques , 2010, EURASIP J. Adv. Signal Process..

[17]  R. Bonney,et al.  Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy , 2009 .

[18]  C. Lintott,et al.  Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey , 2008, 0804.4483.

[19]  John E. O'Reilly,et al.  An algorithm for oceanic front detection in chlorophyll and SST satellite imagery , 2009 .

[20]  Francesca Antonucci,et al.  Automated Image Analysis for the Detection of Benthic Crustaceans and Bacterial Mat Coverage Using the VENUS Undersea Cabled Network , 2011, Sensors.

[21]  Robert A. Sohn,et al.  Assessment of decadal-scale ecological change at a deep Mid-Atlantic hydrothermal vent and reproductive time-series in the shrimp Rimicaris exoculata , 2007, Journal of the Marine Biological Association of the United Kingdom.

[22]  M. O'Neill,et al.  Automated species identification: why not? , 2004, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[23]  Kevin Crowston,et al.  From Conservation to Crowdsourcing: A Typology of Citizen Science , 2011, 2011 44th Hawaii International Conference on System Sciences.

[24]  Peter Arzberger,et al.  New Eyes on the World: Advanced Sensors for Ecology , 2009 .

[25]  Ricardo Serrão Santos,et al.  Distribution and habitat association of benthic fish on the Condor seamount (NE Atlantic, Azores) from in situ observations , 2013 .

[26]  Jenq-Neng Hwang,et al.  Supervised and Unsupervised Feature Extraction Methods for Underwater Fish Species Recognition , 2014, 2014 ICPR Workshop on Computer Vision for Analysis of Underwater Imagery.

[27]  Jacopo Aguzzi,et al.  Diel rhythms in shallow Mediterranean rocky-reef fishes: a chronobiological approach with the help of trained volunteers , 2012, Journal of the Marine Biological Association of the United Kingdom.

[28]  Jacopo Aguzzi,et al.  Diel behavioral rhythms in sablefish (Anoplopoma fimbria) and other benthic species, as recorded by the Deep-sea cabled observatories in Barkley canyon (NEPTUNE-Canada) , 2014 .

[29]  Carsten Rahbek,et al.  Comparing diversity data collected using a protocol designed for volunteers with results from a professional alternative , 2013 .

[30]  Helen E. Roy,et al.  Understanding citizen science and environmental monitoring: final report on behalf of UK Environmental Observation Framework , 2012 .

[31]  Jörg Ontrup,et al.  Use of machine-learning algorithms for the automated detection of cold-water coral habitats: a pilot study , 2009 .

[32]  Maia Hoeberechts,et al.  The Power of Seeing: Experiences using video as a deep-sea engagement and education tool , 2015, OCEANS 2015 - MTS/IEEE Washington.

[33]  Barbara Filipczyk,et al.  Assessing Data Quality , 2018 .

[34]  Malcolm R. Clark,et al.  Mining of deep-sea seafloor massive sulfides: A review of the deposits, their benthic communities, impacts from mining, regulatory frameworks and management strategies , 2013 .

[35]  J. Silvertown.  A new dawn for citizen science. , 2009, Trends in ecology & evolution.

[36]  Roberto Danovaro,et al.  Deep, diverse and definitely different: unique attributes of the world's largest ecosystem , 2010 .

[37]  ZhangZhengyou,et al.  A survey on face detection in the wild , 2015 .

[38]  Jozée Sarrazin,et al.  Elaboration of a video processing platform to analyze the temporal dynamics of hydrothermal ecosystems , 2010 .

[39]  Margaret Kosmala,et al.  Assessing data quality in citizen science (preprint) , 2016, bioRxiv.

[40]  C. L. Van Dover,et al.  Spatial and interannual variation in the faunal distribution at Broken Spur vent field (29°N, Mid-Atlantic Ridge) , 1997 .

[41]  Raimondo Schettini,et al.  Underwater Image Processing: State of the Art of Restoration and Image Enhancement Methods , 2010, EURASIP J. Adv. Signal Process..

[42]  S D Gaines,et al.  From principles to practice: a spatial approach to systematic conservation planning in the deep sea , 2013, Proceedings of the Royal Society B: Biological Sciences.

[43]  Anupam Agrawal,et al.  A survey on activity recognition and behavior understanding in video surveillance , 2012, The Visual Computer.

[44]  J. Gutt,et al.  Semi-Automated Image Analysis for the Assessment of Megafaunal Densities at the Arctic Deep-Sea Observatory HAUSGARTEN , 2012, PloS one.

[45]  Yadvinder Malhi,et al.  Quantifying the sampling error in tree census measurements by volunteers and its effect on carbon stock estimates. , 2013, Ecological applications : a publication of the Ecological Society of America.

[46]  Paolo Menesatti,et al.  Behavioral rhythms of hydrocarbon seep fauna in relation to internal tides , 2010 .

[47]  Tomas J. Bird,et al.  Statistical solutions for error and bias in global citizen science datasets , 2014 .