A computationally efficient pipeline for 3D point cloud reconstruction from video sequences

This paper presents a computationally efficient pipeline to achieve 3D point cloud reconstruction from video sequences. This pipeline involves a key frame selection step to improve the computational efficiency by generating reliable depth information from pair-wise frames. An outlier removal step is then applied in order to further improve the computational efficiency. The reconstruction is achieved based on a new absolute camera pose recovery approach in a computationally efficient manner. This pipeline is devised for both sparse and dense 3D reconstruction. The results obtained from video sequences exhibit higher computational efficiency and lower re-projection errors of the introduced pipeline compared to the existing pipelines.

[1]  Daniel Cremers,et al.  Robust odometry estimation for RGB-D cameras , 2013, 2013 IEEE International Conference on Robotics and Automation.

[2]  David G. Lowe,et al.  Scalable Nearest Neighbor Algorithms for High Dimensional Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Masatoshi Okutomi,et al.  Stable Two View Reconstruction Using the Six-Point Algorithm , 2012, ACCV.

[4]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[5]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[6]  Roland Siegwart,et al.  Versatile distributed pose estimation and sensor self-calibration for an autonomous MAV , 2012, 2012 IEEE International Conference on Robotics and Automation.

[7]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Reinhard Koch,et al.  Automated reconstruction of 3D scenes from sequences of images , 2000 .

[9]  Matthias Nießner,et al.  Combining Inertial Navigation and ICP for Real-time 3D Surface Reconstruction , 2014, Eurographics.

[10]  Horst Bischof,et al.  Efficient structure from motion with weak position and orientation priors , 2011, CVPR 2011 WORKSHOPS.

[11]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[12]  D. Lowe,et al.  Fast Matching of Binary Features , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[13]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[14]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[15]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[16]  Tomás Pajdla,et al.  Omnidirectional Camera Motion Estimation , 2008, VISAPP.

[17]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[18]  Luc Van Gool,et al.  Surviving Dominant Planes in Uncalibrated Structure and Motion Recovery , 2002, ECCV.

[19]  Michal Havlena,et al.  Measuring camera translation by the dominant apical angle , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Juho Kannala,et al.  DT-SLAM: Deferred Triangulation for Robust SLAM , 2014, 2014 2nd International Conference on 3D Vision.

[21]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[22]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[23]  Michael Frankfurter,et al.  Numerical Recipes In C The Art Of Scientific Computing , 2016 .

[24]  S. Shankar Sastry,et al.  c ○ 2000 Kluwer Academic Publishers. Manufactured in The Netherlands. Linear Differential Algorithm for Motion Recovery: A Geometric Approach , 2022 .

[25]  Matthew N. Dailey,et al.  Robust Key Frame Extraction for 3D Reconstruction from Video Streams , 2010, VISAPP.

[26]  David Nistér,et al.  Preemptive RANSAC for live structure and motion estimation , 2005, Machine Vision and Applications.

[27]  Jan-Michael Frahm,et al.  Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[29]  Nasser Kehtarnavaz,et al.  Computationally efficient approach to three-dimensional point cloud reconstruction from video image sequences , 2014, J. Electronic Imaging.

[30]  David W. Murray,et al.  Video-rate Recognition and Localization for Wearable Cameras , 2007, BMVC.

[31]  Paul A. Beardsley,et al.  Sequential Updating of Projective and Affine Structure from Motion , 1997, International Journal of Computer Vision.

[32]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[33]  Manolis I. A. Lourakis,et al.  The design and implementation of a generic sparse bundle adjustment software package based on the Le , 2004 .

[34]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[36]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..