Robust Variational Segmentation of 3D Objects from Multiple Views

We propose a probabilistic formulation of 3D segmentation given a series of images from calibrated cameras. Instead of segmenting each image separately in order to build a 3D surface consistent with these segmentations, we compute the most probable surface that gives rise to the images. Additionally, our method can reconstruct the mean intensity and variance of the extracted object and background. Although it is designed for scenes, where the objects can be distinguished visually from the background (i.e. images of piecewise homogeneous regions), the proposed algorithm can also cope with noisy data. We carry out the numerical implementation in the level set framework. Our experiments on synthetic data sets reveal favorable results compared to state-of-the-art methods, in particular in terms of robustness to noise and initialization.

[1]  R. Cipolla,et al.  A probabilistic framework for space carving , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[2]  Jitendra Malik,et al.  A differential method for computing local shape-from-texture for planar and curved surfaces , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[3]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[5]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[6]  Rachid Deriche,et al.  Active unsupervised texture segmentation on a diffusion based feature space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[7]  Jitendra Malik,et al.  A differential method for computing local shape-from-texture for planar and curved surfaces , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[9]  Daniel Cremers,et al.  Shedding light on stereoscopic segmentation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[10]  Andrew Blake,et al.  Surface shape from the deformation of apparent contours , 1992, International Journal of Computer Vision.

[11]  Jiří Matas,et al.  Computer Vision - ECCV 2004 , 2004, Lecture Notes in Computer Science.

[12]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[13]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[14]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[15]  Pere Brunet,et al.  3D reconstruction with projective octrees and epipolar geometry , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[16]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[17]  Olivier D. Faugeras,et al.  Using Extremal Boundaries for 3-D Object Modeling , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  A. Dervieux,et al.  A finite element method for the simulation of a Rayleigh-Taylor instability , 1980 .

[19]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[20]  R. Rautmann Approximation Methods for Navier-Stokes Problems , 1980 .

[21]  Stefano Soatto,et al.  Stereoscopic Segmentation , 2003, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[22]  Hong Qin,et al.  Shape Reconstruction from 3D and 2D Data Using PDE-Based Deformable Surfaces , 2004, ECCV.

[23]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[24]  Michael Potmesil Generating octree models of 3D objects from their silhouettes in a sequence of images , 1987, Comput. Vis. Graph. Image Process..

[25]  Steven M. Seitz,et al.  Complete scene structure from four point correspondences , 1995, Proceedings of IEEE International Conference on Computer Vision.