Underwater Fish Detection Using Deep Learning for Water Power Applications

Clean energy from oceans and rivers is becoming a reality with the development of new technologies like tidal and instream turbines that generate electricity from naturally flowing water. These new technologies are being monitored for effects on fish and other wildlife using underwater video. Methods for automated analysis of underwater video are needed to lower the costs of analysis and improve accuracy. A deep learning model, YOLO, was trained to recognize fish in underwater video using three very different datasets recorded at real-world water power sites. Training and testing with examples from all three datasets resulted in a mean average precision (mAP) score of 0.5392. To test how well a model could generalize to new datasets, the model was trained using examples from only two of the datasets and then tested on examples from all three datasets. The resulting model could not recognize fish in the dataset that was not part of the training set. The mAP scores on the other two datasets that were included in the training set were higher than the scores achieved by the model trained on all three datasets. These results indicate that different methods are needed in order to produce a trained model that can generalize to new data sets such as those encountered in real world applications.

[1]  Robert B. Fisher,et al.  Detecting, Tracking and Counting Fish in Low Quality Unconstrained Underwater Videos , 2008, VISAPP.

[2]  Dominique Pelletier,et al.  Comparison of visual census and high definition video transects for monitoring coral reef fish assemblages , 2011 .

[3]  B. Polagye,et al.  Environmental Effects of Tidal Energy Development , 2011 .

[4]  Robert B. Fisher,et al.  A research tool for long-term and continuous analysis of fish assemblage in coral-reefs using underwater camera footage , 2014, Ecol. Informatics.

[5]  Sungbin Choi Fish Identification in Underwater Video with Deep Convolutional Neural Network: SNUMedinfo at LifeCLEF Fish task 2015 , 2015, CLEF.

[6]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Xiu Li,et al.  Fast accurate fish detection and recognition of underwater images with Fast R-CNN , 2015, OCEANS 2015 - MTS/IEEE Washington.

[8]  Robert B. Fisher,et al.  Fish4Knowledge: Collecting and Analyzing Massive Coral Reef Fish Video Data , 2016, Intelligent Systems Reference Library.

[9]  Sek M. Chai,et al.  Unsupervised underwater fish detection fusing flow and objectiveness , 2016, 2016 IEEE Winter Applications of Computer Vision Workshops (WACVW).

[10]  Andrea E. Copping,et al.  State of the Science Report: Environmental Effects of Marine Renewable Energy Development Around the World , 2016 .

[11]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Takeshi Naemura,et al.  Learning Multi-frame Visual Representation for Joint Detection and Tracking of Small Objects , 2017, ArXiv.

[13]  S. Matzner,et al.  Studying fish near ocean energy devices using underwater video , 2017, OCEANS 2017 – Anchorage.

[14]  Helmut Mader,et al.  Advances of FishNet towards a fully automatic monitoring system for fish migration , 2017 .

[15]  Hervé Glotin,et al.  LifeCLEF 2017 Lab Overview: Multimedia Species Identification Challenges , 2017, CLEF.

[16]  Sushant Jain,et al.  Underwater Fish Species Classification using Convolutional Neural Network and Deep Learning , 2017, 2017 Ninth International Conference on Advances in Pattern Recognition (ICAPR).

[17]  Bela Stantic,et al.  Assessing fish abundance from underwater video using deep neural networks , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[18]  Frederik Kratzert Fish species classification in underwater video monitoring using Convolutional Neural Networks , 2018 .

[19]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[20]  Simone Marini,et al.  Tracking Fish Abundance by Underwater Image Recognition , 2018, Scientific Reports.

[21]  Joseph Redmon,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.