Autonomous Underwater Monitoring System for Detecting Life on the Seabed by Means of Computer Vision Cloud Services

Autonomous underwater vehicles (AUVs) have increasingly played a key role in monitoring the marine environment, studying its physical-chemical parameters for the supervision of endangered species. AUVs now include a power source and an intelligent control system that allows them to autonomously carry out programmed tasks. Their navigation system is much more challenging than that of land-based applications, due to the lack of connected networks in the marine environment. On the other hand, due to the latest developments in neural networks, particularly deep learning (DL), the visual recognition systems can achieve impressive performance. Computer vision (CV) has especially improved the field of object detection. Although all the developed DL algorithms can be deployed in the cloud, the present cloud computing system is unable to manage and analyze the massive amount of computing power and data. Edge intelligence is expected to replace DL computation in the cloud, providing various distributed, low-latency and reliable intelligent services. This paper proposes an AUV model system designed to overcome latency challenges in the supervision and tracking process by using edge computing in an IoT gateway. The IoT gateway is used to connect the AUV control system to the internet. The proposed model successfully carried out a long-term monitoring mission in a predefined area of shallow water in the Mar Menor (Spain) to track the underwater Pinna nobilis (fan mussel) species. The obtained results clearly justify the proposed system’s design and highlight the cloud and edge architecture performances. They also indicate the need for a hybrid cloud/edge architecture to ensure a real-time control loop for better latency and accuracy to meet the system’s requirements.

[1]  Sajad Saeedi,et al.  AUV Navigation and Localization: A Review , 2014, IEEE Journal of Oceanic Engineering.

[2]  Peter Sincak,et al.  Cloud-Based Object Recognition: A System Proposal , 2013, RiTA.

[3]  Matias Valdenegro-Toro Improving sonar image patch matching via deep learning , 2017, 2017 European Conference on Mobile Robots (ECMR).

[4]  Paolo Casari,et al.  Software-Defined Underwater Acoustic Modems: Historical Review and the NILUS Approach , 2017, IEEE Journal of Oceanic Engineering.

[5]  Marc Chaumont,et al.  A Deep learning method for accurate and fast identification of coral reef fishes in underwater images , 2018, Ecol. Informatics.

[6]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[7]  Jeong Seop Sim,et al.  Smombie Guardian: We watch for potential obstacles while you are walking and conducting smartphone activities , 2018, PloS one.

[8]  Huimin Lu,et al.  Underwater Optical Image Processing: a Comprehensive Review , 2017, Mob. Networks Appl..

[9]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  John J. Leonard,et al.  Navigation for Underwater Vehicles , 2018 .

[11]  Jie Li,et al.  WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images , 2017, IEEE Robotics and Automation Letters.

[12]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Xu Chen,et al.  Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing , 2019, Proceedings of the IEEE.

[14]  Liu Jun,et al.  A review of object detection based on convolutional neural network , 2017, 2017 36th Chinese Control Conference (CCC).

[15]  Weiming Shen,et al.  Internet of Things in Marine Environment Monitoring: A Review , 2019, Sensors.

[16]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[17]  Simone Marini,et al.  Looking inside the Ocean: Toward an Autonomous Imaging System for Monitoring Gelatinous Zooplankton , 2016, Sensors.

[18]  Loris Nanni,et al.  Handcrafted vs. non-handcrafted features for computer vision classification , 2017, Pattern Recognit..

[19]  Vittorio Murino,et al.  A voting-based approach for fast object recognition in underwater acoustic images , 1997 .

[20]  Xiaosu Xu,et al.  Kalman Filter for Cross-Noise in the Integration of SINS and DVL , 2014 .

[21]  Edward N Pugh,et al.  Adaptive algorithms for two-channel polarization sensing under various polarization statistics with nonuniform distributions. , 2006, Applied optics.

[22]  R. K. Hansen An acoustic camera for 3D underwater imaging , 1993 .

[23]  Pål Buhl-Mortensen,et al.  Current and future trends in marine image annotation software , 2016 .

[24]  Carlos Balaguer,et al.  Generation and Processing of Simulated Underwater Images for Infrastructure Visual Inspection with UUVs , 2019, Sensors.

[25]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[26]  Ben. G. Weinstein A computer vision for animal ecology. , 2018, The Journal of animal ecology.

[27]  Ioannis D. Moscholios,et al.  Towards Distributed Data Management in Fog Computing , 2018, Wirel. Commun. Mob. Comput..

[28]  Lei Gao,et al.  Visual Navigation for Recovering an AUV by Another AUV in Shallow Water , 2019, Sensors.

[29]  Changxin Liu,et al.  Nonlinear model predictive dynamic positioning control of an underwater vehicle with an onboard USBL system , 2016 .

[30]  Weisong Shi,et al.  Edge Computing: Vision and Challenges , 2016, IEEE Internet of Things Journal.

[31]  Minh-Duc Hua,et al.  Pool testing of AUV visual servoing for autonomous inspection , 2015 .

[32]  David M. Lane,et al.  Automatic interpretation of sonar imagery using qualitative feature matching , 1994 .

[33]  Francisco Bonnin-Pascual,et al.  On the use of robots and vision technologies for the inspection of vessels: A survey on recent advances , 2019, Ocean Engineering.

[34]  A J Healey,et al.  The Present State of Autonomous Underwater Vehicle (AUV) Applications and Technologies , 2008 .

[35]  Gian Luca Foresti,et al.  A Vision Based System for Object Detection in Underwater Images , 2000, Int. J. Pattern Recognit. Artif. Intell..

[36]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[37]  Daoliang Li,et al.  Integrated navigation for autonomous underwater vehicles in aquaculture: A review , 2020 .

[38]  Kaishun Wu,et al.  Software-Defined Architectures and Technologies for Underwater Wireless Sensor Networks: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[39]  Junaed Sattar,et al.  Toward a Generic Diver-Following Algorithm: Balancing Robustness and Efficiency in Deep Visual Detection , 2018, IEEE Robotics and Automation Letters.

[40]  Rajesh Kannan Megalingam,et al.  Monocular distance estimation using pinhole camera approximation to avoid vehicle crash and back-over accidents , 2016, 2016 10th International Conference on Intelligent Systems and Control (ISCO).

[41]  Dana R. Yoerger,et al.  Autonomous and remotely operated vehicle technology for hydrothermal vent discovery, exploration, and sampling , 2007 .

[42]  Hanumant Singh,et al.  Surveying a Subsea Lava Flow Using the Autonomous Benthic Explorer (abe) , 1998, Int. J. Syst. Sci..

[43]  Anna Goldenberg,et al.  TensorFlow: Biology's Gateway to Deep Learning? , 2016, Cell systems.

[44]  B. Bett,et al.  Autonomous Underwater Vehicles (AUVs): Their past, present and future contributions to the advancement of marine geoscience , 2014 .

[45]  Phil F. Culverhouse,et al.  Automated identification of benthic epifauna with computer vision , 2019, Marine Ecology Progress Series.

[46]  Diego Alonso,et al.  An Autonomous Solar-Powered Marine Robotic Observatory for Permanent Monitoring of Large Areas of Shallow Water , 2018, Sensors.

[47]  Ahmed El Oualkadi,et al.  Smart Industrial IoT Monitoring and Control System Based on UAV and Cloud Computing Applied to a Concrete Plant , 2019, Sensors.

[48]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Paul Ozog,et al.  Long‐term Mapping Techniques for Ship Hull Inspection and Surveillance using an Autonomous Underwater Vehicle , 2016, J. Field Robotics.

[50]  Ming Du,et al.  Computer vision algorithms and hardware implementations: A survey , 2019, Integr..

[51]  Unnikrishnan V. Painumgal,et al.  Autonomous Underwater Vehicle for Vision Based Tracking , 2018 .

[52]  Marouane Salhaoui,et al.  Comparative Analysis of Energy Efficiency and Latency of Fog and Cloud Architectures , 2019, 2019 International Conference on Sensing and Instrumentation in IoT Era (ISSI).

[53]  Oscar Pizarro,et al.  High‐Resolution Underwater Robotic Vision‐Based Mapping and Three‐Dimensional Reconstruction for Archaeology , 2017, J. Field Robotics.

[54]  Yoav Y. Schechner,et al.  Regularized Image Recovery in Scattering Media , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Carlos Silvestre,et al.  TRIDENT: A Framework for Autonomous Underwater Intervention Missions with Dexterous Manipulation Capabilities , 2010 .

[56]  Xiaofei Wang,et al.  Convergence of Edge Computing and Deep Learning: A Comprehensive Survey , 2019, IEEE Communications Surveys & Tutorials.