3D reconstruction system based on incremental structure from motion using a camera with varying parameters

In this paper, we present a flexible and fast system for multi-scale objects/scenes 3D reconstruction from uncalibrated images/video taken by a moving camera characterized by variable parameters. The proposed system is based on incremental structure from motion and good exploitation of bundle adjustment. At first, from two selected images, our system allows to recover, in a well-chosen reference, coordinates of a set of 3D points. In this context, we have proposed a new method of self-calibration based on the use of two unknown scene points with their image projections. After that, new images are inserted progressively using 3D information already obtained. Local bundle adjustment is used to adjust the new estimated entities. At some time, we introduce a global bundle adjustment to adjust as best as possible all estimated entities and to have an initial 3D model of quality covering an interesting part of the object/scene. This model will be used as reference for the insertion of the rest of images. The proposed system allows to obtain satisfactory results within a reasonable time.

[1]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[2]  Abderrahim Saaidi,et al.  Incremental Multi-view 3D Reconstruction Starting from Two Images Taken by a Stereo Pair of Cameras , 2015 .

[3]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[4]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[5]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[6]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[8]  Long Quan,et al.  A quasi-dense approach to surface reconstruction from uncalibrated images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Pengpeng Zhao,et al.  A Comparative Study of SIFT and its Variants , 2013 .

[10]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[11]  Abderrahim Saaidi,et al.  Reconstruction of 3D Scenes by Camera Self-Calibration and Using Genetic Algorithms , 2016 .

[12]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Marshall W. Bern,et al.  A new Voronoi-based surface reconstruction algorithm , 1998, SIGGRAPH.

[14]  Pascal Fua,et al.  Efficient large-scale multi-view stereo for ultra high-resolution image sets , 2011, Machine Vision and Applications.

[15]  Long Quan,et al.  Match Propagation for Image-Based Modeling and Rendering , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Larry S. Davis,et al.  3D Surface Reconstruction Using Graph Cuts with Surface Constraints , 2006, ECCV.

[18]  J. J. Moré,et al.  Levenberg--Marquardt algorithm: implementation and theory , 1977 .

[19]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Chi Fang,et al.  3D face sparse reconstruction based on local linear fitting , 2013, The Visual Computer.

[21]  Changchang Wu,et al.  Critical Configurations for Radial Distortion Self-Calibration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  H. Jin Kim,et al.  Real-time 6-DOF monocular visual SLAM in a large-scale environment , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[24]  K. L. Chan,et al.  3D object model reconstruction from image sequence based on photometric consistency in volume space , 2009, Pattern Analysis and Applications.

[25]  Simon Fuhrmann,et al.  MVE - An image-based reconstruction environment , 2015, Comput. Graph..

[26]  Michel Dhome,et al.  Generic and real-time structure from motion using local bundle adjustment , 2009, Image Vis. Comput..

[27]  Matthew A. Brown,et al.  Unsupervised 3D object recognition and reconstruction in unordered datasets , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[28]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[29]  Luc Van Gool,et al.  Dense matching of multiple wide-baseline views , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[30]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[31]  Abderrahim Saaidi,et al.  3D face reconstruction using images from cameras with varying parameters , 2017, Int. J. Autom. Comput..

[32]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[33]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision: Algorithm Evaluation and Error Analysis , 2004 .

[34]  Guanghui Wang,et al.  Perspective 3-D Euclidean Reconstruction With Varying Camera Parameters , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Zhaoqi Wang,et al.  3D entity-based stereo matching with ground control points and joint second-order smoothness prior , 2014, The Visual Computer.

[36]  Jan Mayer,et al.  A numerical evaluation of preprocessing and ILU-type preconditioners for the solution of unsymmetric sparse linear systems using iterative methods , 2009, TOMS.

[37]  Jean-Philippe Pons,et al.  High Accuracy and Visibility-Consistent Dense Multiview Stereo , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.