A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation

We investigate the feasibility of reconstructing an arbitrarily-shaped specular scene (refractive or mirror-like) from one or more viewpoints. By reducing shape recovery to the problem of reconstructing individual 3D light paths that cross the image plane, we obtain three key results. First, we show how to compute the depth map of a specular scene from a single viewpoint, when the scene redirects incoming light just once. Second, for scenes where incoming light undergoes two refractions or reflections, we show that three viewpoints are sufficient to enable reconstruction in the general case. Third, we show that it is impossible to reconstruct individual light paths when light is redirected more than twice. Our analysis assumes that, for every point on the image plane, we know at least one 3D point on its light path. This leads to reconstruction algorithms that rely on an "environment matting" procedure to establish pixel-to-point correspondences along a light path. Preliminary results for a variety of scenes (mirror, glass, etc) are also presented

[1]  Pietro Perona,et al.  Local Analysis for 3D Reconstruction of Specular Surfaces - Part II , 2002, ECCV.

[2]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[3]  Olivier D. Faugeras,et al.  Complete Dense Stereovision Using Level Set Methods , 1998, ECCV.

[4]  Andrew S. Glassner,et al.  Principles of Digital Image Synthesis , 1995 .

[5]  Hans-Peter Seidel,et al.  3D acquisition of mirroring objects using striped patterns , 2005, Graph. Model..

[6]  Shree K. Nayar,et al.  A Theory of Specular Surface Geometry , 2004, International Journal of Computer Vision.

[7]  Andrew Blake,et al.  Specular Stereo , 1985, IJCAI.

[8]  Lee E. Weiss,et al.  Structured Highlight Inspection of Specular Surfaces , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Kiriakos N. Kutulakos,et al.  Dynamic Refraction Stereo , 2005, ICCV.

[10]  Shree K. Nayar,et al.  What does motion reveal about transparency? , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  Peter F. Sturm,et al.  Voxel carving for specular surfaces , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  R. Pless,et al.  Discrete and differential two-view constraints for general imaging systems , 2002, Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in conjunction with ECCV'02.

[13]  Pau Gargallo,et al.  General Specular Surface Triangulation , 2006, ACCV.

[14]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  B. Gotwols,et al.  Two-dimensional optical measurement of wave slope. , 1983, Applied optics.

[16]  Katsushi Ikeuchi,et al.  Transparent surface modeling from a pair of polarization images , 2004 .

[17]  Katsushi Ikeuchi,et al.  Determining Surface Orientations of Specular Surfaces by Using the Photometric Stereo Method , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Eron Steger.  Reconstructing Transparent Objects by Refractive Light-Path Triangulation , 2006 .

[19]  Shree K. Nayar,et al.  Planar catadioptric stereo: geometry and calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[20]  Derek Bradley,et al.  Tomographic reconstruction of transparent objects , 2006, SIGGRAPH '06.

[21]  Hans-Gerd Maas,et al.  New developments in Multimedia Photogrammetry , 2001 .

[22]  J. Koenderink,et al.  The structure of two-dimensional scalar fields with applications to vision , 1979, Biological Cybernetics.

[23]  Kiriakos N. Kutulakos,et al.  Photo-consistent 3D fire by Flame-Sheet decomposition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Mitsuru Baba,et al.  New laser rangefinder for three-dimensional shape measurement of specular objects , 2001 .

[25]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[26]  B. Jähne,et al.  Imaging of short ocean wind waves: a critical theoretical review , 1994 .

[27]  Wojciech Matusik,et al.  Acquisition and Rendering of Transparent and Refractive Objects , 2002, Rendering Techniques.

[28]  Jiang Yu Zheng,et al.  Acquiring a Complete 3D Model from Specular Motion under the Illumination of Circular-Shaped Light Sources , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  David J. Kriegman,et al.  On Refractive Optical Flow , 2004, ECCV.

[30]  C. Cox,et al.  Measuring the two-dimensional structure of a wavy water surface optically: A surface gradient detector , 1994 .

[31]  A. Okamoto.  Orientation problem of two-media photographs with curved boundary surfaces , 1984 .

[32]  Brian A. Barsky,et al.  Reconstructing curved surfaces from specular reflection patterns using spline surface fitting of normals , 1996, SIGGRAPH.

[33]  J. Hecksher-Sørensen,et al.  Optical Projection Tomography as a Tool for 3D Microscopy and Gene Expression Studies , 2002, Science.

[34]  Hiroshi Murase.  Surface shape reconstruction of an undulating transparent object , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[35]  Joachim Höhle.  Reconstruction of the Underwater Object , 1971 .

[36]  Steven M. Seitz,et al.  Shape and materials by example: a photometric stereo approach , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[37]  Kristin J. Dana,et al.  A novel approach for texture shape recovery , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[38]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[39]  Shree K. Nayar,et al.  A general imaging model and a method for finding its parameters , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.