Vision System of Mobile Robot Combining Binocular and Depth Cameras

In order to optimize the three-dimensional (3D) reconstruction and obtain more precise actual distances of the object, a 3D reconstruction system combining binocular and depth cameras is proposed in this paper. The whole system consists of two identical color cameras, a TOF depth camera, an image processing host, a mobile robot control host, and a mobile robot. Because of structural constraints, the resolution of TOF depth camera is very low, which difficultly meets the requirement of trajectory planning. The resolution of binocular stereo cameras can be very high, but the effect of stereo matching is not ideal for low-texture scenes. Hence binocular stereo cameras also difficultly meet the requirements of high accuracy. In this paper, the proposed system integrates depth camera and stereo matching to improve the precision of the 3D reconstruction. Moreover, a double threads processing method is applied to improve the efficiency of the system. The experimental results show that the system can effectively improve the accuracy of 3D reconstruction, identify the distance from the camera accurately, and achieve the strategy of trajectory planning.

[1]  Abdalla A. Osman,et al.  Distance alert system using Stereo vision and feature extraction , 2017, 2017 International Conference on Communication, Control, Computing and Electronics Engineering (ICCCCEE).

[2]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Yo-Sung Ho,et al.  Disparity map generation for color image using TOF depth camera , 2011, 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[4]  P. Pharpatara,et al.  3-D Trajectory Planning of Aerial Vehicles Using RRT* , 2017, IEEE Transactions on Control Systems Technology.

[5]  John J. Leonard,et al.  Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age , 2016, IEEE Transactions on Robotics.

[6]  Dadet Pramadihanto,et al.  FloW vision: Depth image enhancement by combining stereo RGB-depth sensor , 2016, 2016 International Conference on Knowledge Creation and Intelligent Computing (KCIC).

[7]  In-So Kweon,et al.  Time-of-Flight Sensor Calibration for a Color and Depth Camera Pair , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  K. S. Venkatesh,et al.  Accurate rough terrain modeling from fused 3D point cloud data , 2015, 2015 Annual IEEE India Conference (INDICON).

[9]  Hanqing Lu,et al.  Fast and Accurate Image Matching with Cascade Hashing for 3D Reconstruction , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Dwi Pebrianti,et al.  Study on 3D scene reconstruction in robot navigation using stereo vision , 2016, 2016 IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS).

[11]  Sangyoun Lee,et al.  Registration method between ToF and color cameras for face recognition , 2011, 2011 6th IEEE Conference on Industrial Electronics and Applications.

[12]  Emanuele Menegatti,et al.  Unsupervised intrinsic and extrinsic calibration of a camera-depth sensor couple , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Li Shi,et al.  Theory and experiment analysis of disparity for stereoscopic image pairs , 2001, Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing. ISIMP 2001 (IEEE Cat. No.01EX489).

[14]  Silong Peng,et al.  A Practical Roadside Camera Calibration Method Based on Least Squares Optimization , 2014, IEEE Transactions on Intelligent Transportation Systems.

[15]  Radu Horaud,et al.  High-resolution depth maps based on TOF-stereo fusion , 2012, 2012 IEEE International Conference on Robotics and Automation.

[16]  Azra Fetic,et al.  The procedure of a camera calibration using Camera Calibration Toolbox for MATLAB , 2012, 2012 Proceedings of the 35th International Convention MIPRO.

[17]  Yo-Sung Ho,et al.  High-quality multi-view depth generation using multiple color and depth cameras , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[18]  Xiaochun Cao,et al.  Camera Calibration Using Symmetric Objects , 2006, IEEE Transactions on Image Processing.

[19]  Yaonan Wang,et al.  Registration and fusion for ToF camera and 2D camera reading , 2013, 2013 Chinese Automation Congress.

[20]  Xiangyang Wang,et al.  Image-based 3D models reconstruction , 2011 .

[21]  A. Aydin Alatan,et al.  3D tracking using visual and range data , 2012, 2012 20th Signal Processing and Communications Applications Conference (SIU).

[22]  Wei Liu,et al.  Parking Like a Human: A Direct Trajectory Planning Solution , 2017, IEEE Transactions on Intelligent Transportation Systems.

[23]  Masatoshi Okutomi,et al.  Calibration and rectification for reflection stereo , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.