Full 3D reconstruction of transparent objects

Numerous techniques have been proposed for reconstructing 3D models for opaque objects in past decades. However, none of them can be directly applied to transparent objects. This paper presents a fully automatic approach for reconstructing complete 3D shapes of transparent objects. Through positioning an object on a turntable, its silhouettes and light refraction paths under different viewing directions are captured. Then, starting from an initial rough model generated from space carving, our algorithm progressively optimizes the model under three constraints: surface and refraction normal consistency, surface projection and silhouette consistency, and surface smoothness. Experimental results on both synthetic and real objects demonstrate that our method can successfully recover the complex shapes of transparent objects and faithfully reproduce their light refraction properties.

[1]  Jingyi Yu,et al.  Reconstructing Gas Flows Using Light-Path Approximation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Hans-Peter Seidel,et al.  Polarization and Phase-Shifting for 3D Scanning of Translucent Objects , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[4]  Ruigang Yang,et al.  Interactive Visual Hull Refinement for Specular and Transparent Object Surface Reconstruction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Zhong Zhou,et al.  Reconstruction of three-dimensional flame with color temperature , 2014, The Visual Computer.

[6]  Marcus A. Magnor,et al.  Image-based tomographic reconstruction of flames , 2004, SIGGRAPH '04.

[7]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[8]  H. Seidel,et al.  Fluorescent immersion range scanning , 2008, ACM Trans. Graph..

[9]  Minglun Gong,et al.  Frequency-Based Environment Matting by Compressive Sensing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  Daniel Cohen-Or,et al.  Parameterization-free projection for geometry reconstruction , 2007, ACM Trans. Graph..

[11]  David Salesin,et al.  Environment matting extensions: towards higher accuracy and real-time capture , 2000, SIGGRAPH.

[12]  Derek Bradley,et al.  Tomographic reconstruction of transparent objects , 2006, SIGGRAPH '06.

[13]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[14]  Anthony B. Davis,et al.  Airborne Three-Dimensional Cloud Tomography , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Kiriakos N. Kutulakos,et al.  Reconstructing the Surface of Inhomogeneous Transparent Scenes by Scatter-Trace Photography , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[16]  Sameer Agarwal,et al.  Refractive height fields from single and multiple images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Anthony B. Davis,et al.  Multiple-Scattering Microphysics Tomography , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Kei Iwasaki,et al.  Poisson-Based Continuous Surface Generation for Goal-Based Caustics , 2014, ACM Trans. Graph..

[19]  Daniel Cohen-Or,et al.  Edge-aware point set resampling , 2013, ACM Trans. Graph..

[20]  Cong Phuoc Huynh,et al.  Shape and refractive index recovery from single-view polarisation images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Konrad Schindler,et al.  Massively Parallel Multiview Stereopsis by Surface Normal Diffusion , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Daniel Cohen-Or,et al.  Quality-driven poisson-guided autoscanning , 2014, ACM Trans. Graph..

[23]  Hans-Peter Seidel,et al.  Time-resolved 3d capture of non-stationary gas flows , 2008, SIGGRAPH Asia '08.

[24]  M. Schmid.  Principles Of Optics Electromagnetic Theory Of Propagation Interference And Diffraction Of Light , 2016 .

[25]  Kiriakos N. Kutulakos,et al.  Dynamic Refraction Stereo , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Abhijeet Ghosh,et al.  Acquiring Axially-Symmetric Transparent Objects Using Single-View Transmission Imaging , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Wolfgang Heidrich,et al.  Stochastic tomography and its applications in 3D imaging of mixing fluids , 2012, ACM Trans. Graph..

[28]  Gordon Wetzstein,et al.  Refractive shape from light field distortion , 2011, 2011 International Conference on Computer Vision.

[29]  Yasushi Yagi,et al.  Recovering Transparent Shape from Time-of-Flight Distortion , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Bastian Goldlücke,et al.  4D imaging through spray-on optics , 2017, ACM Trans. Graph..

[31]  Pierre Alliez,et al.  State of the Art in Surface Reconstruction from Point Clouds , 2014, Eurographics.

[32]  Paolo Cignoni,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Efficient and Flexible Sampling with Blue Noise Properties of Triangular Meshes , 2022 .

[33]  Pieter Peers,et al.  Wavelet Environment matting , 2003, Rendering Techniques.

[34]  Qionghai Dai,et al.  Recovering Scene Geometry under Wavy Fluid via Distortion and Defocus Analysis , 2014, ECCV.

[35]  Katsushi Ikeuchi,et al.  Inverse polarization raytracing: estimating surface shapes of transparent objects , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  Tai-Pang Wu,et al.  Adequate reconstruction of transparent objects on a shoestring budget , 2011, CVPR 2011.

[37]  Matthias Zwicker,et al.  Deep points consolidation , 2015, ACM Trans. Graph..

[38]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[39]  Minglun Gong,et al.  3D Reconstruction of Transparent Objects with Position-Normal Consistency , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Daniel Cohen-Or,et al.  Consolidation of unorganized point clouds for surface reconstruction , 2009, ACM Trans. Graph..

[41]  Jianfei Cai,et al.  Compressive environment matting , 2014, The Visual Computer.

[42]  Kiriakos N. Kutulakos,et al.  A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[43]  Qiang Zhou,et al.  Dip transform for 3D shape reconstruction , 2017, ACM Trans. Graph..

[44]  Andrea Tagliasacchi,et al.  High-contrast computational caustic design , 2014, ACM Trans. Graph..

[45]  Andrew W. Fitzgibbon,et al.  Image-based environment matting , 2002, SIGGRAPH '02.

[46]  Aswin C. Sankaranarayanan,et al.  What does a single light-ray reveal about a transparent object? , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[47]  Kiriakos N. Kutulakos,et al.  Transparent and Specular Object Reconstruction , 2010, Comput. Graph. Forum.

[48]  Minglun Gong,et al.  Stereo-Based 3D Reconstruction of Dynamic Fluid Surfaces by Global Optimization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Shree K. Nayar,et al.  What does motion reveal about transparency? , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[50]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Jan Kautz,et al.  Polarimetric Multi-view Stereo , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Gabriel Taubin,et al.  A benchmark for surface reconstruction , 2013, TOGS.