Range and Vision Sensors Fusion for Outdoor 3D Reconstruction

The conscience of the surrounding environment is inevitable task for several applications such as mapping, autonomous navigation and localization. In this paper we are interested by exploiting the complementarity of a panoramic microwave radar and a monocular camera for 3D reconstruction of large scale environments. Considering the robustness to environmental conditions and depth detection ability of the radar on one hand, and the high spatial resolution of a vision sensor on the other hand, makes these tow sensors well adapted for large scale outdoor cartography. Firstly, the system model of the two sensors is represented and a new 3D reconstruction method based on sensors geometry is introduced. Secondly, we address the global calibration problem which consists in finding the exact transformation between radar and camera coordinate systems. The method is based on the optimization of a non-linear criterion obtained from a set of radar-to-image target correspondences. Both methods have been validated with synthetic and real data.

[1]  Corina Kim Schindhelm,et al.  Evaluating SLAM Approaches for Microsoft Kinect , 2012, ICWMC 2012.

[2]  Pietro Cerri,et al.  Radar-vision fusion for vehicle detection , 2006 .

[3]  Andrea Fossati,et al.  Consumer Depth Cameras for Computer Vision , 2013, Advances in Computer Vision and Pattern Recognition.

[4]  P. Sturm.  A method for 3D reconstruction of piecewise planar objects from single panoramic images , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[5]  Tomás Pajdla,et al.  3D with Kinect , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[6]  M. Okutomi,et al.  Obstacle detection using millimeter-wave radar and its visualization on image sequence , 2004, ICPR 2004.

[7]  P. Cerri,et al.  Obstacle detection and classification fusing radar and vision , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[8]  Nanning Zheng,et al.  Integrating Millimeter Wave Radar with a Monocular Vision Sensor for On-Road Obstacle Detection Applications , 2011, Sensors.

[9]  M. Skolnik,et al.  Introduction to Radar Systems , 2021, Advances in Adaptive Radar Detection and Range Estimation.

[10]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[11]  Lang Hong,et al.  Fusion of Doppler Radar and video information for automated traffic surveillance , 2009, 2009 12th International Conference on Information Fusion.

[12]  Ernst D. Dickmanns,et al.  Radar and vision data fusion for hybrid adaptive cruise control on highways , 2001, Machine Vision and Applications.

[13]  M.O. Monod,et al.  High resolution mapping of the environment with a ground-based radar imager , 2009, 2009 International Radar Conference "Surveillance for a Safer World" (RADAR 2009).

[14]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[15]  Primo Zingaretti,et al.  Complete classification of raw LIDAR data and 3D reconstruction of buildings , 2006, Pattern Analysis and Applications.

[16]  Jan-Michael Frahm,et al.  Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Dwipen Bhagawati,et al.  PHOTOGRAMMETRY AND 3 D RECONSTRUCTION – THE STATE OF THE ART , 2000 .

[18]  T. O. Jones,et al.  Automotive radar: A brief review , 1974 .

[19]  Alberto Broggi,et al.  Vehicle and Guard Rail Detection Using Radar and Vision Data Fusion , 2007, IEEE Transactions on Intelligent Transportation Systems.

[20]  Rama Chellappa,et al.  Structure from Motion Using Sequential Monte Carlo Methods , 2004, International Journal of Computer Vision.

[21]  Michel Dhome,et al.  Monocular Vision for Mobile Robot Localization and Autonomous Navigation , 2007, International Journal of Computer Vision.

[22]  Anton Kummert,et al.  Radar-vision fusion for vehicle detection by means of improved haar-like feature and AdaBoost approach , 2007, 2007 15th European Signal Processing Conference.

[23]  Emre Ertin,et al.  Sparse Signal Methods for 3-D Radar Imaging , 2011, IEEE Journal of Selected Topics in Signal Processing.