Automatic annotation of coral reefs using deep learning

Healthy coral reefs play a vital role in maintaining biodiversity in tropical marine ecosystems. Deep sea exploration and imaging have provided us with a great opportunity to look into the vast and complex marine ecosystems. Data acquisition from the coral reefs has facilitated the scientific investigation of these intricate ecosystems. Millions of digital images of the sea floor have been collected with the help of Remotely Operated Vehicles (ROVs) and Autonomous Underwater Vehicles (AUVs). Automated technology to monitor the health of the oceans allows for transformational ecological outcomes by standardizing methods for detecting and identifying species. Manual annotation is a tediously repetitive and a time consuming task for marine experts. It takes 10-30 minutes for a marine expert to meticulously annotate a single image. This paper aims to automate the analysis of large available AUV imagery by developing advanced deep learning tools for rapid and large-scale automatic annotation of marine coral species. Such an automated technology would greatly benefit marine ecological studies in terms of cost, speed, accuracy and thus in better quantifying the level of environmental change marine ecosystems can tolerate. We propose a deep learning based classification method for coral reefs. We also report the application of the proposed technique towards the automatic annotation of unlabelled mosaics of the coral reef in the Abrolhos Islands, Western Australia. Our proposed method automatically quantifies the coral coverage in this region and detects a decreasing trend in coral population which is in line with conclusions by marine ecologists.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Stefan B. Williams,et al.  Variable Responses of Benthic Communities to Anomalously Warm Sea Temperatures on a High-Latitude Coral Reef , 2014, PloS one.

[3]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Nancy Knowlton,et al.  Climate change impacts on marine ecosystems. , 2012, Annual review of marine science.

[5]  R. Grosberg,et al.  Climate Change, Human Impacts, and the Resilience of Coral Reefs , 2003, Science.

[6]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[7]  Robert B. Fisher,et al.  Coral classification with hybrid feature representations , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[8]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Stefan B. Williams,et al.  Australian sea-floor survey data, with images and expert annotations , 2015, Scientific Data.

[10]  Rafael Garcia,et al.  Automated classification and thematic mapping of bacterial mats in the North Sea , 2013, 2013 MTS/IEEE OCEANS - Bergen.

[11]  Svetlana Lazebnik,et al.  Multi-scale Orderless Pooling of Deep Convolutional Activation Features , 2014, ECCV.

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  E. Barbier,et al.  Response to Comments on "Impacts of Biodiversity Loss on Ocean Ecosystem Services" , 2007, Science.

[14]  Grant B. Deane,et al.  Automated processing of coral reef benthic images , 2009 .

[15]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Stefan B. Williams,et al.  Monitoring of Benthic Reference Sites: Using an Autonomous Underwater Vehicle , 2012, IEEE Robotics & Automation Magazine.

[17]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[18]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[19]  Mohammed Bennamoun,et al.  A Discriminative Representation of Convolutional Features for Indoor Scene Recognition , 2015, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[20]  O. Pizarro,et al.  Towards image-based marine habitat classification , 2008, OCEANS 2008.

[21]  R. Steneck,et al.  Coral Reefs Under Rapid Climate Change and Ocean Acidification , 2007, Science.

[22]  Robert B. Fisher,et al.  A research tool for long-term and continuous analysis of fish assemblage in coral-reefs using underwater camera footage , 2014, Ecol. Informatics.

[23]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[24]  David J. Kriegman,et al.  Automated annotation of coral reef survey images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.