Reconstructing Reflective and Transparent Surfaces from Epipolar Plane Images

While multi-view stereo reconstruction of Lambertian surfaces is nowadays highly robust, reconstruction methods based on correspondence search usually fail in the presence of ambiguous information, like in the case of partially reflecting and transparent surfaces. On the epipolar plane images of a 4D light field, however, surfaces like these give rise to overlaid patterns of oriented lines. We show that these can be identified and analyzed quickly and accurately with higher order structure tensors. The resulting method can reconstruct with high precision both the geometry of the surface as well as the geometry of the reflected or transmitted object. Accuracy and feasibility are shown on both ray-traced synthetic scenes and real-world data recorded by our gantry.

[1]  Neill W Campbell,et al.  IEEE International Conference on Computer Vision and Pattern Recognition , 2008 .

[2]  Daniel Cremers,et al.  Global Solutions of Variational Models with Convex Regularization , 2010, SIAM J. Imaging Sci..

[3]  Michael Goesele,et al.  Image-based rendering for scenes with reflections , 2012, ACM Trans. Graph..

[4]  Oliver K. Smith,et al.  Eigenvalues of a symmetric 3 × 3 matrix , 1961, Commun. ACM.

[5]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[6]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Anat Levin,et al.  User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior , 2004, ECCV.

[8]  Steven M. Seitz,et al.  Shape and Spatially-Varying BRDFs from Photometric Stereo , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Ruigang Yang,et al.  BRDF Invariant Stereo Using Light Transport Constancy , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[11]  Stefano Soatto,et al.  Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance , 2005, International Journal of Computer Vision.

[12]  Y. Weiss,et al.  Separating reflections from a single image using local features , 2004, CVPR 2004.

[13]  Reinhard Klein,et al.  Heightfield and spatially varying BRDF Reconstruction for Materials with Interreflections , 2009, Comput. Graph. Forum.

[14]  H. Knutsson,et al.  Estimating Multiple Depths in Semi-transparent Stereo Images , 1999 .

[15]  Richard Szeliski,et al.  Stereo matching with linear superposition of layers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  J. Bigun,et al.  Optimal Orientation Detection of Linear Symmetry , 1987, ICCV 1987.

[18]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Til Aach,et al.  Analysis of Superimposed Oriented Patterns , 2006, IEEE Transactions on Image Processing.

[20]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.