The Pinax-model for accurate and efficient refraction correction of underwater cameras in flat-pane housings

Abstract The calibration and refraction correction process for underwater cameras with flat-pane interfaces is presented that is very easy and convenient to use in real world applications while yielding very accurate results. The correction is derived from an analysis of the axial camera model for underwater cameras, which is among others computationally hard to tackle. It is shown how realistic constraints on the distance of the camera to the window can be exploited, which leads to an approach dubbed Pinax Model as it combines aspects of a virtual pinhole model with the projection function from the axial camera model. It allows the pre-computation of a lookup-table for very fast refraction correction of the flat-pane with high accuracy. The model takes the refraction indices of water into account, especially with respect to salinity, and it is therefore sufficient to calibrate the underwater camera only once in air. It is demonstrated by real world experiments with several underwater cameras in different salt and sweet water conditions that the proposed process outperforms standard methods. Among others, it is shown how the presented method leads to accurate results with single in-air calibration and even with just estimated salinity values.

[1]  Marc Carreras,et al.  Navigating and mapping with the SPARUS AUV in a natural and unstructured underwater environment , 2011, OCEANS'11 MTS/IEEE KONA.

[2]  Shahriar Negahdaripour,et al.  Opti-Acoustic Stereo Imaging, System Calibration and 3-D Reconstruction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Stefan B. Williams,et al.  Bundle adjustment in large-scale 3D reconstructions based on underwater robotic surveys , 2011, OCEANS 2011 IEEE - Spain.

[4]  Anne Jordt,et al.  Refractive Calibration of Underwater Cameras , 2012, ECCV.

[5]  E. Malis,et al.  3D Reconstruction of Natural Underwater Scenes Using the Stereovision System IRIS , 2007, OCEANS 2007 - Europe.

[6]  Stefan B. Williams,et al.  Error modeling and calibration of exteroceptive sensors for accurate mapping applications , 2010 .

[7]  S. Negahdaripour,et al.  Integrated System for Robust 6-DOF Positioning Utilizing New Closed-Form Visual Motion Estimation Methods in Planar Terrains , 2006, IEEE Journal of Oceanic Engineering.

[8]  Marc Pollefeys,et al.  CamOdoCal: Automatic intrinsic and extrinsic calibration of a rig with multiple generic cameras and odometry , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Visesh Chari,et al.  A theory of multi-layer flat refractive geometry , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  José Santos-Victor,et al.  Underwater Video Mosaics as Visual Navigation Maps , 2000, Comput. Vis. Image Underst..

[11]  R. W. Austin,et al.  The Index of Refraction of Seawater , 1976 .

[12]  R. L. Marks,et al.  Automatic visual station keeping of an underwater robot , 1994, Proceedings of OCEANS'94.

[13]  Minglun Gong,et al.  Underwater Camera Calibration Using Wavelength Triangulation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[15]  Hanumant Singh,et al.  Stereo self-calibration for seafloor mapping using AUVs , 2010, 2010 IEEE/OES Autonomous Underwater Vehicles.

[16]  Julien Hué,et al.  Underwater Archaeological 3D Surveys Validation within the Removed Sets Framework , 2011, ECSQARU.

[17]  Andreas Birk,et al.  Underwater place recognition in noisy stereo data using FAB-MAP with a multimodal vocabulary from 2D texture and 3D surface descriptors , 2015, OCEANS 2015 - Genova.

[18]  Hanumant Singh,et al.  Relative Pose Estimation for Instrumented, Calibrated Imaging Platforms , 2003, DICTA.

[19]  Marcel Rothenbeck,et al.  Use of REMUS 6000 AUVs in the search for the Air France Flight 447 , 2011, OCEANS'11 MTS/IEEE KONA.

[20]  P. Firoozfam,et al.  An ROV Stereovision System for Ship-Hull Inspection , 2006, IEEE Journal of Oceanic Engineering.

[21]  Mark Hardy,et al.  Autonomous inspection using an underwater 3D LiDAR , 2013, 2013 OCEANS - San Diego.

[22]  Marie-José Aldon,et al.  Camera Self-Calibration in Underwater Environment , 2003, WSCG.

[23]  Refractive Structure-from-Motion on Underwater Images , 2013, 2013 IEEE International Conference on Computer Vision.

[24]  M. de Groot,et al.  Benthic habitat mapping with autonomous underwater vehicles , 2008, OCEANS 2008.

[25]  Tamaki Ura,et al.  Wide area 3D seafloor reconstruction and its application to sea fauna density mapping , 2013, 2013 OCEANS - San Diego.

[26]  Daniel Toal,et al.  Review of Machine Vision Applications in Unmanned Underwater Vehicles , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[27]  Anne Jordt,et al.  Refractive 3D reconstruction on underwater images , 2016 .

[28]  L. Brignone,et al.  First sea trials of a laser aided three dimensional underwater image mosaicing technique , 2011, OCEANS 2011 IEEE - Spain.

[29]  R. C. Millard,et al.  An index of refraction algorithm for seawater over temperature, pressure, salinity, density, and wavelength , 1990 .

[30]  Marc Pollefeys,et al.  Infrastructure-based calibration of a multi-camera rig , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Takashi Matsuyama,et al.  A Pixel-Wise Varifocal Camera Model for Efficient Forward Projection and Linear Extrinsic Calibration of Underwater Cameras with Flat Housings , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[32]  Steven Lake Waslander,et al.  Underwater stereo SLAM with refraction correction , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Visesh Chari,et al.  Multi-View Geometry of the Refractive Plane , 2009, BMVC.

[34]  Atsushi Yamashita,et al.  Underwater sensing with omni-directional stereo camera , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[35]  Yee-Hong Yang,et al.  Experimental study of the influence of refraction on underwater three-dimensional reconstruction using the SVP camera model. , 2012, Applied optics.

[36]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Minglun Gong,et al.  Refractive Epipolar Geometry for Underwater Stereo Matching , 2011, 2011 Canadian Conference on Computer and Robot Vision.

[38]  Pierre Drap,et al.  We All Live in a Virtual Submarine , 2010, IEEE Computer Graphics and Applications.

[39]  Gian Luca Foresti,et al.  Visual inspection of sea bottom structures by an autonomous underwater vehicle , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[40]  Emanuele Trucco,et al.  Application of 2 1/2 D visual servoing to underwater vehicle station-keeping , 2000, OCEANS 2000 MTS/IEEE Conference and Exhibition. Conference Proceedings (Cat. No.00CH37158).

[41]  Yee-Hong Yang,et al.  Two-View Camera Housing Parameters Calibration for Multi-layer Flat Refractive Interface , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Geoffrey A. Hollinger,et al.  Uncertainty-driven view planning for underwater inspection , 2012, 2012 IEEE International Conference on Robotics and Automation.

[43]  Ryan M. Eustice,et al.  Real-Time Visual SLAM for Autonomous Underwater Hull Inspection Using Visual Saliency , 2013, IEEE Transactions on Robotics.

[44]  J. Kojima,et al.  Autonomous underwater vehicle AQUA EXPLORER 2 for inspection of underwater cables , 2000, Proceedings of the 2000 International Symposium on Underwater Technology (Cat. No.00EX418).

[45]  Pere Ridao,et al.  Coverage path planning with realtime replanning for inspection of 3D underwater structures , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[46]  Anne Jordt,et al.  Calibration of Housing Parameters for Underwater Stereo-Camera Rigs , 2011, BMVC.

[47]  Roland Siegwart,et al.  Infrastructure-based calibration of a multi-camera rig , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[48]  E. Fry,et al.  Empirical equation for the index of refraction of seawater. , 1995, Applied optics.

[49]  Rongxin Li,et al.  Quantitative photogrammetric analysis of digital underwater video imagery , 1997 .

[50]  Hanumant Singh,et al.  Robotic tools for deep water archaeology: Surveying an ancient shipwreck with an autonomous underwater vehicle , 2010, J. Field Robotics.

[51]  Reinhard Koch,et al.  3D reconstruction based on underwater video from ROV Kiel 6000 considering underwater imaging conditions , 2009, OCEANS 2009-EUROPE.

[52]  Euan S. Harvey,et al.  Design and Calibration of an Underwater Stereo-video System for the Monitoring of Marine Fauna Populations , 2001 .

[53]  Jean-Thierry Lapresté,et al.  Dry camera calibration for underwater applications , 2003, Machine Vision and Applications.

[54]  Shahriar Negahdaripour,et al.  Undersea optical stationkeeping: Improved methods , 1991, J. Field Robotics.

[55]  H. Singh,et al.  Hemispherical refraction and camera calibration in underwater vision , 2008, OCEANS 2008.

[56]  Anne Jordt,et al.  Refractive Plane Sweep for Underwater Images , 2013, GCPR.