Towards Full 3D Helmholtz Stereovision Algorithms

Helmholtz stereovision methods are limited to binocular stereovision or depth maps reconstruction. In this paper, we extend these methods to recover the full 3D shape of the objects of a scene from multiview Helmholtz stereopsis. Thus, we are able to reconstruct the complete three-dimensional shape of objects made of any arbitrary and unknown bidirectional reflectance distribution function. Unlike previous methods, this can be achieved using a full surface representation model. In particular occlusions (self occlusions as well as cast shadows) are easier to handle in the surface optimization process. More precisely, we use a triangular mesh representation which allows to naturally specify relationships between the geometry of a point of the scene and its surface normal. We show how to implement the presented approach using a coherent gradient descent flow. Results and benefits are illustrated on various examples.

[1]  Olivier D. Faugeras,et al.  Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score , 2007, International Journal of Computer Vision.

[2]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[3]  Michael Goesele,et al.  Multi-View Stereo Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[4]  Peter F. Sturm,et al.  Joint Estimation of Shape and Reflectance using Multiple Images with Known Illumination Conditions , 2009, International Journal of Computer Vision.

[5]  Jean-Philippe Pons,et al.  Towards high-resolution large-scale multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Michel Barlaud,et al.  Using the Shape Gradient for Active Contour Segmentation: from the Continuous to the Discrete Formulation , 2006, Journal of Mathematical Imaging and Vision.

[7]  David J. Kriegman,et al.  Beyond Lambert: reconstructing specular surfaces using color , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Daniel Cremers,et al.  Shedding light on stereoscopic segmentation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[9]  Peter H. Tu,et al.  Surface reconstruction via Helmholtz reciprocity with a single image pair , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Steven M. Seitz,et al.  Example-based photometric stereo: shape reconstruction with general, varying BRDFs , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Roberto Cipolla,et al.  Multiview Photometric Stereo , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  David J. Kriegman,et al.  Color Subspaces as Photometric Invariants , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[13]  Todd E. Zickler.  Reciprocal Image Features for Uncalibrated Helmholtz Stereopsis , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[14]  Jean-Philippe Pons,et al.  Minimizing the Multi-view Stereo Reprojection Error for Triangular Surface Meshes , 2008, BMVC.

[15]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  David J. Kriegman,et al.  Binocular Helmholtz stereopsis , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Tianli Yu,et al.  Recovering shape and reflectance model of non-lambertian objects from multiple views , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[18]  Stefano Soatto,et al.  Multi-view stereo beyond Lambert , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  Jean-Yves Guillemaut,et al.  Helmholtz Stereopsis on rough and strongly textured surfaces , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[20]  Pau Gargallo I. Piracés.  Contributions to the Bayesian Approach to Multi-View Stereo. (Contributions à l'approche bayésienne pour la stéréovision multi-vues) , 2008 .

[21]  Jean-Philippe Pons,et al.  Delaunay Deformable Models: Topology-Adaptive Meshes Based on the Restricted Delaunay Triangulation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Daniel Cremers,et al.  3-D Reconstruction of Shaded Objects from Multiple Images Under Unknown Illumination , 2008, International Journal of Computer Vision.

[23]  Jean-Philippe Pons,et al.  Generalized Surface Flows for Mesh Processing , 2007 .

[24]  Radu Horaud,et al.  TransforMesh : A Topology-Adaptive Mesh-Based Approach to Surface Evolution , 2007, ACCV.