Dense 3D reconstruction combining depth and RGB information

Dense 3D reconstruction has important applications in many fields. The existing depth information based methods are typically constrained in their effective camera-object distance which should be from 0.4m to 4m. We present a novel method that can achieve a more accurate dense 3D reconstruction with an RGB-D camera when the distance between the camera and object is less than 0.4m, which enlarges the application range. Our approach combines a depth information based 3D model method with a RGB information based method to refine the reconstruction results when the camera fails to acquire the correct depth information. Rich RGB information captured from a color camera along with feature detection and triangulation methods are used to obtain accurate camera poses and 3D points when the camera is close to the object. Compared with the reconstruction results obtained from depth information only, quantitative experimental results show that our method is more effective, particularly when the camera is close to the object in the scene.

[1]  LiangRonghua,et al.  Bayesian multi-distribution-based discriminative feature extraction for 3D face recognition , 2015 .

[2]  Moongu Jeon,et al.  Data fusion of radar and image measurements for multi-object tracking via Kalman filtering , 2014, Inf. Sci..

[3]  Meng Wang,et al.  Indoor scene understanding via monocular RGB-D images , 2015, Inf. Sci..

[4]  Gamini Dissanayake,et al.  A robust RGB-D SLAM algorithm , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[6]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[7]  Fuchun Sun,et al.  Recursive depth parametrization of monocular visual navigation: Observability analysis and performance evaluation , 2014, Inf. Sci..

[8]  YouJane,et al.  Multi-view ensemble manifold regularization for 3D object recognition , 2015 .

[9]  Li Yu,et al.  Beyond the interference problem: hierarchical patterns for multiple-projector structured light system. , 2014, Applied optics.

[10]  Xu Wang,et al.  A bundled-optimization model of multiview dense depth map synthesis for dynamic scene reconstruction , 2015, Inf. Sci..

[11]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[12]  Ling Shao,et al.  Building recognition in urban environments: A survey of state-of-the-art and future challenges , 2014, Inf. Sci..

[13]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[14]  Andrés Iglesias,et al.  Particle swarm optimization for non-uniform rational B-spline surface reconstruction from clouds of 3D data points , 2012, Inf. Sci..

[15]  Luc Van Gool,et al.  In-hand scanning with online loop closure , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[16]  Jun Yu,et al.  Multi-view ensemble manifold regularization for 3D object recognition , 2015, Inf. Sci..

[17]  Xuelong Li,et al.  GA-SIFT: A new scale invariant feature transform for multispectral image using geometric algebra , 2014, Inf. Sci..

[18]  David Valiente,et al.  A modified stochastic gradient descent algorithm for view-based SLAM using omnidirectional images , 2014, Inf. Sci..

[19]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[20]  Zicheng Liu,et al.  Real time gaze estimation with a consumer depth camera , 2015, Inf. Sci..

[21]  Wang Cheng,et al.  Registration Using Multiplanar Structures for Augmented Reality Systems , 2008, J. Comput. Inf. Sci. Eng..

[22]  Li Yu,et al.  Depth map reconstruction and rectification through coding parameters for mobile 3D video system , 2015, Neurocomputing.

[23]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[24]  ChangHwan Kim,et al.  A 3D-grasp synthesis algorithm to grasp unknown objects based on graspable boundary and convex segments , 2015, Inf. Sci..

[25]  Qi Tian,et al.  Task-Dependent Visual-Codebook Compression , 2012, IEEE Transactions on Image Processing.

[26]  Xu Wang,et al.  A multi-dimensional image quality prediction model for user-generated images in social networks , 2014, Inf. Sci..

[27]  Ke Lu,et al.  3D model retrieval and classification by semi-supervised learning with content-based similarity , 2014, Inf. Sci..

[28]  Meng Wang,et al.  Towards efficient support relation extraction from RGBD images , 2015, Inf. Sci..

[29]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[30]  Wen Gao,et al.  Location Discriminative Vocabulary Coding for Mobile Landmark Search , 2011, International Journal of Computer Vision.

[31]  Francisco R. Feito-Higueruela,et al.  A new approach to create textured urban models through genetic algorithms , 2013, Inf. Sci..

[32]  Jan-Michael Frahm,et al.  Modeling and Recognition of Landmark Image Collections Using Iconic Scene Graphs , 2008, International Journal of Computer Vision.

[33]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[34]  Diego Viejo,et al.  Combining visual features and Growing Neural Gas networks for robotic 3D SLAM , 2014, Inf. Sci..

[35]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[36]  Hao Liu,et al.  Dense depth image synthesis via energy minimization for three-dimensional video , 2015, Signal Process..

[37]  Luming Zhang,et al.  Recognizing multi-view objects with occlusions using a deep architecture , 2015, Inf. Sci..

[38]  Dan Xia,et al.  Handling occlusions in augmented reality based on 3D reconstruction method , 2015, Neurocomputing.

[39]  Kok-Lim Low Linear Least-Squares Optimization for Point-to-Plane ICP Surface Registration , 2004 .

[40]  Xindong Wu,et al.  3-D Object Retrieval With Hausdorff Distance Learning , 2014, IEEE Transactions on Industrial Electronics.

[41]  Michel Dhome,et al.  Real Time Localization and 3D Reconstruction , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[42]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[43]  Wen Gao,et al.  Learning to Distribute Vocabulary Indexing for Scalable Visual Search , 2013, IEEE Transactions on Multimedia.

[44]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[45]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR 2011.

[46]  Zhengjun Zha,et al.  Gradient-domain-based enhancement of multi-view depth video , 2014, Inf. Sci..

[47]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[48]  Slobodan Ilic,et al.  RGB-D camera-based parallel tracking and meshing , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[49]  Mohammed Bennamoun,et al.  A novel local surface feature for 3D object recognition under clutter and occlusion , 2015, Inf. Sci..

[50]  LiuZicheng,et al.  Real time gaze estimation with a consumer depth camera , 2015 .

[51]  Rongrong Ji,et al.  Robust infrared target tracking based on particle filter with embedded saliency detection , 2015, Inf. Sci..