Progressive 3D Model Acquisition with a Commodity Hand-Held Camera

We present a system for progressive 3D model acquisition with a commodity hand-held camera. The pipeline starts with a real-time scanning stage accomplished using a sparse point-based tracker for camera pose estimation and a dense patch-based tracker for dense reconstruction. While the user scans the target object, a model composed of local planar patches is continuously updated and displayed for visual feedback. This live feedback loop allows the user to choose new viewpoints based on the state of the current reconstruction, and determine if the model is completely covered in desired details. After live scanning is completed, our system refines the reconstructed patches into denser and more accurate patches through an offline model refinement procedure. We demonstrate the ability of our system on various real datasets.

[1]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  Jan-Michael Frahm,et al.  Real-Time Visibility-Based Fusion of Depth Maps , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[3]  Tom Drummond,et al.  ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition , 2009, BMVC.

[4]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[5]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[6]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[7]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[8]  Jonathan Balzer,et al.  CLAM: Coupled Localization and Mapping with Efficient Outlier Handling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Marc Pollefeys,et al.  Live Metric 3D Reconstruction on Mobile Phones , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[11]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[12]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[13]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[14]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[15]  Andrew J. Davison,et al.  Live dense reconstruction with a single moving camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Jean-Philippe Pons,et al.  Towards high-resolution large-scale multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[18]  Marc Pollefeys,et al.  Multi-View Stereo via Graph Cuts on the Dual of an Adaptive Tetrahedral Mesh , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[21]  C. Zach.  Fast and High Quality Fusion of Depth Maps , 2008 .

[22]  Thomas Brox,et al.  Dense 3D Reconstruction with a Hand-Held Camera , 2012, DAGM/OAGM Symposium.