Simultaneous 3D Reconstruction for Water Surface and Underwater Scene

This paper presents the first approach for simultaneously recovering the 3D shape of both the wavy water surface and the moving underwater scene. A portable camera array system is constructed, which captures the scene from multiple viewpoints above the water. The correspondences across these cameras are estimated using an optical flow method and are used to infer the shape of the water surface and the underwater scene. We assume that there is only one refraction occurring at the water interface. Under this assumption, two estimates of the water surface normals should agree: one from Snell’s law of light refraction and another from local surface structure. The experimental results using both synthetic and real data demonstrate the effectiveness of the presented approach.

[1]  C. Cox,et al.  Measuring the two-dimensional structure of a wavy water surface optically: A surface gradient detector , 1994 .

[2]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[3]  David Salesin,et al.  Environment matting extensions: towards higher accuracy and real-time capture , 2000, SIGGRAPH.

[4]  Feng Li,et al.  Dynamic fluid surface acquisition using a camera array , 2011, 2011 International Conference on Computer Vision.

[5]  Kiriakos N. Kutulakos,et al.  A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation , 2005, ICCV.

[6]  Qionghai Dai,et al.  Recovering Scene Geometry under Wavy Fluid via Distortion and Defocus Analysis , 2014, ECCV.

[7]  C. Mulsow,et al.  A FLEXIBLE MULTI-MEDIA BUNDLE APPROACH , 2010 .

[8]  Minglun Gong,et al.  Frequency-Based Environment Matting by Compressive Sensing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Visesh Chari,et al.  A theory of multi-layer flat refractive geometry , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Gordon Wetzstein,et al.  Hand-held Schlieren Photography with Light Field probes , 2011, 2011 IEEE International Conference on Computational Photography (ICCP).

[11]  Anne Jordt,et al.  Calibration of Housing Parameters for Underwater Stereo-Camera Rigs , 2011, BMVC.

[12]  Feng Li,et al.  Angular domain reconstruction of dynamic 3D fluid surfaces , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Minglun Gong,et al.  Full 3D reconstruction of transparent objects , 2018, ACM Trans. Graph..

[14]  H. Saito,et al.  3D shape measurement of underwater objects using motion stereo , 1995, Proceedings of IECON '95 - 21st Annual Conference on IEEE Industrial Electronics.

[15]  Abhijeet Ghosh,et al.  Acquiring Axially-Symmetric Transparent Objects Using Single-View Transmission Imaging , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Minglun Gong,et al.  3D Reconstruction of Transparent Objects with Position-Normal Consistency , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jesse Belden,et al.  Calibration of multi-camera systems with refractive interfaces , 2013 .

[18]  Wolfgang Heidrich,et al.  From capture to simulation , 2014, ACM Trans. Graph..

[19]  Stuart N. Lane,et al.  REMOTE SENSING OF CLEAR-WATER, SHALLOW, GRAVEL-BED RIVERS USING DIGITAL PHOTOGRAMMETRY , 2001 .

[20]  Yasushi Yagi,et al.  Recovering Transparent Shape from Time-of-Flight Distortion , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Alexei A. Efros,et al.  Seeing through water , 2004, NIPS.

[22]  Yuandong Tian,et al.  Seeing through water: Image restoration using model-based tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[23]  James T. Kajiya,et al.  Ray tracing complex scenes , 1986, SIGGRAPH.

[24]  Jingyi Yu,et al.  Reconstructing Gas Flows Using Light-Path Approximation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Yoav Y. Schechner,et al.  Triangulation in random refractive distortions , 2013, ICCP.

[26]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.

[27]  David J. Kriegman,et al.  Photometric Stereo in a Scattering Medium , 2015, ICCV.

[28]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[29]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  L. Dagum,et al.  OpenMP: an industry standard API for shared-memory programming , 1998 .

[31]  Ernst Rank,et al.  Image-based mesh generation of tubular geometries under circular motion in refractive environments , 2018, Machine Vision and Applications.

[32]  Minglun Gong,et al.  Underwater Camera Calibration Using Wavelength Triangulation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Minglun Gong,et al.  Stereo-Based 3D Reconstruction of Dynamic Fluid Surfaces by Global Optimization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Sameer Agarwal,et al.  Refractive height fields from single and multiple images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  B. Jähne,et al.  Imaging of short ocean wind waves: a critical theoretical review , 1994 .

[36]  Gadi Katzir,et al.  Striking of underwater prey by a reef heron,Egretta gularis schistacea , 1987, Journal of Comparative Physiology.

[37]  A. Adamson,et al.  Ray tracing point set surfaces , 2003, 2003 Shape Modeling International..

[38]  Kai Han,et al.  A fixed viewpoint approach for dense reconstruction of transparent objects , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Hiroshi Murase,et al.  Surface Shape Reconstruction of a Nonrigid Transport Object Using Refraction and Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Tsuhan Chen,et al.  Multi-view 3D reconstruction for scenes under the refractive plane with known vertical direction , 2011, 2011 International Conference on Computer Vision.

[41]  João Paulo Costeira,et al.  Stereo Reconstruction of a Submerged Scene , 2005, IbPRIA.

[42]  Imari Sato,et al.  Shape from Water: Bispectral Light Absorption for Depth Recovery , 2016, ECCV.

[43]  Frédo Durand,et al.  Refraction Wiggles for Measuring Fluid Depth and Velocity from Video , 2014, ECCV.

[44]  Kiriakos N. Kutulakos,et al.  Dynamic Refraction Stereo , 2005, ICCV.

[45]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.