Real-Time Underwater StereoFusion

Many current and future applications of underwater robotics require real-time sensing and interpretation of the environment. As the vast majority of robots are equipped with cameras, computer vision is playing an increasingly important role it this field. This paper presents the implementation and experimental results of underwater StereoFusion, an algorithm for real-time 3D dense reconstruction and camera tracking. Unlike KinectFusion on which it is based, StereoFusion relies on a stereo camera as its main sensor. The algorithm uses the depth map obtained from the stereo camera to incrementally build a volumetric 3D model of the environment, while simultaneously using the model for camera tracking. It has been successfully tested both in a lake and in the ocean, using two different state-of-the-art underwater Remotely Operated Vehicles (ROVs). Ongoing work focuses on applying the same algorithm to acoustic sensors, and on the implementation of a vision based monocular system with the same capabilities.

[1]  Steven Lake Waslander,et al.  Underwater stereo SLAM with refraction correction , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Horst Bischof,et al.  Online 3D reconstruction using convex optimization , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[3]  Pere Ridao,et al.  Autonomous underwater panel operation by GIRONA500 UVMS: A practical approach to autonomous underwater manipulation , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[5]  Yi Dong,et al.  A sonar image mosaicing algorithm based on improved SIFT for USV , 2014, 2014 IEEE International Conference on Mechatronics and Automation.

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  Olaf Kähler,et al.  A Framework for the Volumetric Integration of Depth Images , 2014, ArXiv.

[8]  E. Omerdic,et al.  OceanRINGS: System concept & applications , 2012, 2012 20th Mediterranean Conference on Control & Automation (MED).

[9]  B. He,et al.  3D reconstruction model of underwater environment in stereo vision system , 2013, 2013 OCEANS - San Diego.

[10]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[11]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Paul A. Beardsley,et al.  Underwater 3D capture using a low-cost commercial depth camera , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[13]  Paul Ozog,et al.  Pose-Graph SLAM for Underwater Navigation , 2017 .

[14]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[15]  Andrew Hogue,et al.  Underwater environment reconstruction using stereo and inertial data , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[16]  Dean Brown,et al.  Decentering distortion of lenses , 1966 .

[17]  F. Torre.  3-D reconstruction of biological objects using underwater video technique and image processing , 2003 .

[18]  Benedetto Allotta,et al.  A low cost autonomous underwater vehicle for patrolling and monitoring , 2017 .

[19]  Fabio Bruno,et al.  An Alignment Method for the Integration of Underwater 3D Data Captured by a Stereovision System and an Acoustic Camera , 2016, Sensors.

[20]  Daniel Toal,et al.  Closing the gap between industrial robots and underwater manipulators , 2015, OCEANS 2015 - MTS/IEEE Washington.

[21]  Edin Omerdic,et al.  Fully automatic visual servoing control for work-class marine intervention ROVs , 2018 .

[22]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[23]  Shahram Izadi,et al.  MonoFusion: Real-time 3D reconstruction of small scenes with a single web camera , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[24]  David Scaradozzi,et al.  Real-time reconstruction of underwater environments: From 2D to 3D , 2015, OCEANS 2015 - MTS/IEEE Washington.

[25]  Ron Lewis.  Sensing and Control for Autonomous Vehicles: Applications to Land, Water and Air Vehicles , 2018 .

[26]  J. Elvander,et al.  ROVs and AUVs in support of marine renewable technologies , 2012, 2012 Oceans.

[27]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[28]  Fabrice Mériaudeau,et al.  Underwater 3-D Scene Reconstruction Using Kinect v2 Based on Physical Models for Refraction and Time of Flight Correction , 2017, IEEE Access.

[29]  Kok-Lim Low.  Linear Least-Squares Optimization for Point-to-Plane ICP Surface Registration , 2004 .

[30]  Ozgur Yilmaz,et al.  Stereo and kinect fusion for continuous 3D reconstruction and visual odometry , 2013, 2013 International Conference on Electronics, Computer and Computation (ICECCO).

[31]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[32]  Gabriel Oliver,et al.  Stereo SLAM for robust dense 3D reconstruction of underwater environments , 2015, OCEANS 2015 - Genova.

[33]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Huimin Lu,et al.  Depth Map Reconstruction for Underwater Kinect Camera Using Inpainting and Local Image Mode Filtering , 2017, IEEE Access.

[35]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[36]  Sebastian Thrun,et al.  Robotic mapping: a survey , 2003 .

[37]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[38]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[39]  Peter-Pike J. Sloan,et al.  Interactive ray tracing for isosurface rendering , 1998, Proceedings Visualization '98 (Cat. No.98CB36276).

[40]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[41]  Massimo Caccia,et al.  A survey on real-time motion estimation techniques for underwater robots , 2014, Journal of Real-Time Image Processing.

[42]  Zlatko Franjcic,et al.  Underwater reconstruction using depth sensors , 2014, SIGGRAPH ASIA Technical Briefs.

[43]  Emanuele Trucco,et al.  A compact algorithm for rectification of stereo pairs , 2000, Machine Vision and Applications.

[44]  F. Diaz Ledezma,et al.  A Market Survey of Offshore Underwater Robotic Inspection Technologies for the Oil and Gas Industry , 2015 .

[45]  Olaf Kähler,et al.  Very High Frame Rate Volumetric Integration of Depth Images on Mobile Devices , 2015, IEEE Transactions on Visualization and Computer Graphics.

[46]  Pere Ridao,et al.  I-AUV Mechatronics Integration for the TRIDENT FP7 Project , 2015, IEEE/ASME Transactions on Mechatronics.

[47]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[48]  Junku Yuh,et al.  Underwater Robots , 2012, Springer Handbook of Robotics, 2nd Ed..

[49]  Andrew J. Davison,et al.  Live dense reconstruction with a single moving camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[50]  Pierre Drap,et al.  We All Live in a Virtual Submarine , 2010, IEEE Computer Graphics and Applications.

[51]  Joaquim Salvi,et al.  Real-time mosaicing with two-dimensional forward-looking sonar , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[52]  P. Firoozfam,et al.  An ROV Stereovision System for Ship-Hull Inspection , 2006, IEEE Journal of Oceanic Engineering.