Dense 3D reconstruction from specularity consistency

In this work, we consider the dense reconstruction of specular objects. We propose the use of a specularity constraint, based on surface normal/depth consistency, to define a matching cost function that can drive standard stereo reconstruction methods. We discuss the types of ambiguity that can arise, and suggest an aggregation method based on anisotropic diffusion that is particularly suitable for this matching cost function. We also present a controlled illumination setup that includes a pair of cameras and one LCD monitor, which is used as a calibrated, variable-position light source. We use this setup to evaluate the proposed method on real data, and demonstrate its capacity to recover high-quality depth and orientation from specular objects.

[1]  E. North Coleman,et al.  Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry , 1982, Comput. Graph. Image Process..

[2]  Richard Szeliski,et al.  Stereo Matching with Nonlinear Diffusion , 1998, International Journal of Computer Vision.

[3]  Kiriakos N. Kutulakos,et al.  Reconstructing the Surface of Inhomogeneous Transparent Scenes by Scatter-Trace Photography , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[4]  In-So Kweon,et al.  Correspondence Search in the Presence of Specular Highlights Using Specular-Free Two-Band Images , 2006, ACCV.

[5]  Lee E. Weiss,et al.  Structured Highlight Inspection of Specular Surfaces , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[7]  Jiang Yu Zheng,et al.  Acquiring a Complete 3D Model from Specular Motion under the Illumination of Circular-Shaped Light Sources , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Shree K. Nayar,et al.  A Theory of Specular Surface Geometry , 2004, International Journal of Computer Vision.

[9]  Stephen Lin,et al.  Multibaseline stereo in the presence of specular reflections , 2002, Object recognition supported by user interaction for service robots.

[10]  Andrew Blake,et al.  The information available to a moving observer from specularities , 1989, Image Vis. Comput..

[11]  Hans-Peter Seidel,et al.  3D acquisition of mirroring objects using striped patterns , 2005, Graph. Model..

[12]  Ohad Ben-Shahar,et al.  Toward a Theory of Shape from Specular Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Ruigang Yang,et al.  BRDF Invariant Stereo Using Light Transport Constancy , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Andrew Gardner,et al.  Linear light source reflectometry , 2003, ACM Trans. Graph..

[15]  Szymon Rusinkiewicz,et al.  Improved sub-pixel stereo correspondences through symmetric refinement , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Andrew Blake,et al.  Geometry From Specularities , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[17]  Hans-Peter Seidel,et al.  Mesostructure from Specularity , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Andrew Blake,et al.  Specular Stereo , 1985, IJCAI.

[19]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[20]  Peter F. Sturm,et al.  Voxel carving for specular surfaces , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  Terrance E. Boult,et al.  Constraining Object Features Using a Polarization Reflectance Model , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Szymon Rusinkiewicz,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, ACM Trans. Graph..

[23]  Pau Gargallo,et al.  General Specular Surface Triangulation , 2006, ACCV.

[24]  Henrik Aanæs,et al.  A variational analysis of shape from specularities using sparse data , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[25]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[26]  Shree K. Nayar,et al.  Stereo and Specular Reflection , 1998, International Journal of Computer Vision.

[27]  Shree K. Nayar,et al.  Separation of Reflection Components Using Color and Polarization , 1997, International Journal of Computer Vision.

[28]  Stephen Lin,et al.  Diffuse-Specular Separation and Depth Recovery from Image Sequences , 2002, ECCV.

[29]  Mohan M. Trivedi,et al.  Real-time range image segmentation using adaptive kernels and Kalman filtering , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[30]  Wei Zhou,et al.  Binocular Stereo Dense Matching in the Presence of Specular Reflections , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[31]  Takeo Kanade,et al.  Determining shape and reflectance of hybrid surfaces by photometric sampling , 1989, IEEE Trans. Robotics Autom..

[32]  Min Chen,et al.  Local Shape from Mirror Reflections , 2005, International Journal of Computer Vision.

[33]  Kiriakos N. Kutulakos,et al.  A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[34]  Shree K. Nayar,et al.  A theory of multiplexed illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[35]  Brian A. Barsky,et al.  Reconstructing curved surfaces from specular reflection patterns using spline surface fitting of normals , 1996, SIGGRAPH.

[36]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[37]  Olga Veksler,et al.  Markov random fields with efficient approximations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).