Improving PMVS Algorithm for 3D Scene Reconstruction from Sparse Stereo Pairs

3D scene reconstruction resulting from a limited number of stereo pairs captured by a 3D camera is a nontrivial and challenging task even for current state-of-the-art multi-view stereo (MVS) reconstruction algorithms. It also has many application potentials in related techniques, such as robotics, virtual reality, video games, and 3D animation. In this paper, we analyze the performance of the PMVS (Patch-based Multi-View Stereo software) for scene reconstruction from stereo pairs of scenes captured by a simple 3D camera. We demonstrate that when applied to a limited number of stereo pairs, PMVS is inadequate for 3D scene reconstruction and discuss new strategies to overcome these limitations to improve 3D reconstruction. The proposed Canny edge feature-based PMVS algorithm is shown to produce better reconstruction results. We also discuss further enhancements using dense feature matching and disparity map-based stereo reconstruction.

[1]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[2]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[3]  Jean Ponce,et al.  Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[5]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[7]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, CVPR.

[8]  Luc Van Gool,et al.  Visual modelling: from images to images , 2002, Comput. Animat. Virtual Worlds.

[9]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[11]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[12]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Kostas Daniilidis,et al.  Multi-camera reconstruction based on surface normal estimation and best viewpoint selection , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .