Bayesian Stochastic Mesh Optimization for 3D reconstruction

We describe a mesh based approach to the problem of structure from motion. The input to the algorithm is a small set of images, sparse noisy feature correspondences (such as those provided by a Harris corner detector and cross correlation) and the camera geometry plus calibration. The output is a 3D mesh, that when projected onto each view, is visually consistent with the images. There are two contributions in this paper. The first is a Bayesian formulation in which simplicity and smoothness assumptions are encoded in the prior distribution. The resulting posterior is optimized by simulated annealing. The second and more important contribution is a way to make this optimization scheme more efficient. Generic simulated annealing has been long studied in computer vision and is thought to be highly inefficient. This is often because the proposal distribution searches regions of space which are far from the modes. In order to improve the performance of simulated annealing it has long been acknowledged that choice of the correct proposal distribution is of paramount importance to convergence. Taking inspiration from RANSAC andimportance sampling we craft a proposal distribution that is tailored to the problem of structure from motion. This makes our approach particularly robust to noise and ambiguity. We show results for an artificial object and an architectural scene.

[1]  Pascal Fua,et al.  Object-centered surface reconstruction: Combining multi-image stereo and shading , 1995, International Journal of Computer Vision.

[2]  Greg Turk,et al.  Image-Driven Mesh Optimization , 2000 .

[3]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[4]  Michael Garland,et al.  Simplifying surfaces with color and texture using quadric error metrics , 1998, IEEE Visualization.

[5]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[7]  G. M. Seed,et al.  Mesh optimisation , 2001 .

[8]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[9]  Stan Sclaroff,et al.  Stochastic mesh-based multiview reconstruction , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[10]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[11]  Armin Gruen,et al.  Videometrics and Optical Methods for 3d Shape Measurement , 2000 .

[12]  Roberto Cipolla,et al.  Combining single view recognition and multiple view stereo for architectural scenes , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Hugues Hoppe,et al.  New quadric metric for simplifying meshes with appearance attributes , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[14]  Marc Pollefeys,et al.  Multiple view geometry , 2005 .

[15]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[16]  Steven M. Seitz,et al.  Image-based multiresolution shape recovery by surface deformation , 2000, IS&T/SPIE Electronic Imaging.

[17]  Michael Garland,et al.  Simplifying surfaces with color and texture using quadric error metrics , 1998, Proceedings Visualization '98 (Cat. No.98CB36276).

[18]  Takeo Kanade,et al.  Image-consistent surface triangulation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).