3D Reconstruction from Accidental Motion

We have discovered that 3D reconstruction can be achieved from asingle still photographic capture due to accidental motions of thephotographer, even while attempting to hold the camera still. Although these motions result in little baseline and therefore high depth uncertainty, in theory, we can combine many such measurements over the duration of the capture process (a few seconds) to achieve usable depth estimates. Wepresent a novel 3D reconstruction system tailored for this problemthat produces depth maps from short video sequences from standard cameraswithout the need for multi-lens optics, active sensors, or intentionalmotions by the photographer. This result leads to the possibilitythat depth maps of sufficient quality for RGB-D photography applications likeperspective change, simulated aperture, and object segmentation, cancome "for free" for a significant fraction of still photographsunder reasonable conditions.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  C. Tomasi Detection and Tracking of Point Features , 1991 .

[3]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.

[5]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Stéphane Christy,et al.  Euclidean Shape and Motion from Multiple Perspective Views by Affine Iterations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  John Oliensis Computing the camera heading from multiple frames , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Richard Szeliski,et al.  Vision Algorithms: Theory and Practice , 2002, Lecture Notes in Computer Science.

[11]  Stefano Soatto,et al.  Optimal Structure from Motion: Local Ambiguities and Global Estimates , 2004, International Journal of Computer Vision.

[12]  John Oliensis,et al.  A Multi-Frame Structure-from-Motion Algorithm under Perspective Projection , 1999, International Journal of Computer Vision.

[13]  John Oliensis The Least-Squares Error for Structure from Infinitesimal Motion , 2004, ECCV.

[14]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision: N-View Geometry , 2004 .

[15]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[16]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[17]  Fredrik Kahl,et al.  Multiple view geometry and the L/sub /spl infin//-norm , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[18]  F. Kahl Multiple View Geometry and the L-infinity Norm , 2005, ICCV 2005.

[19]  S. Shankar Sastry,et al.  The Recursive Multi-Frame Planar Parallax Algorithm , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[20]  David J. Kriegman,et al.  Practical Global Optimization for Multiview Geometry , 2006, ECCV.

[21]  Richard I. Hartley,et al.  Recovering Camera Motion Using L\infty Minimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[23]  Pushmeet Kohli,et al.  Robust Higher Order Potentials for Enforcing Label Consistency , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Andrew W. Fitzgibbon,et al.  Global stereo reconstruction under second order smoothness priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Roberto Cipolla,et al.  Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo , 2008, ECCV.

[26]  Nikos Komodakis,et al.  Beyond pairwise energies: Efficient optimization for higher-order MRFs , 2009, CVPR.

[27]  Andrew Adams,et al.  Fast High‐Dimensional Filtering Using the Permutohedral Lattice , 2010, Comput. Graph. Forum.

[28]  Carlos Hernández,et al.  Shape from Photographs: A Multi-view Stereo Pipeline , 2010, Computer Vision: Detection, Recognition and Reconstruction.

[29]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR.

[30]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR 2011.

[31]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[32]  Carsten Rother,et al.  Fast cost-volume filtering for visual correspondence and beyond , 2011, CVPR 2011.

[33]  Yongduek Seo,et al.  Verifying Global Minima for L2 Minimization Problems in Multiple View Geometry , 2012, International Journal of Computer Vision.

[34]  Daniel Wagner,et al.  User friendly SLAM initialization , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).