Online Intelligent Calibration of Cameras and LiDARs for Autonomous Driving Systems*

The online extrinsic parameters calibration among multiple sensors for data fusion has been a crucial task in autonomous driving. This paper proposes a new online, robust pipeline of automatic extrinsic parameters calibration between monocular RGB cameras and LiDARs for autonomous driving applications. The novelties of this work includes threefold: (1) an automatic full pipeline of the calibration is developed for the autonomous driving system; (2) an initialization stage is developed for calibration without the initial value of extrinsic parameters; (3) a combination of depth estimation and edge detection is used to improve the calibration accuracy and robustness. We implement the proposed calibration algorithm on an autonomous driving platform, a BYD Qin electric sedan. Both the simulation and experiment results verify the online and robust performance of the proposed calibration pipeline.

[1]  Qi Hao,et al.  Development of a virtual reality teleconference system using distributed depth sensors , 2016, 2016 2nd IEEE International Conference on Computer and Communications (ICCC).

[2]  Dacheng Tao,et al.  Deep Ordinal Regression Network for Monocular Depth Estimation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  A. E. Eiben,et al.  Directed Locomotion for Modular Robots with Evolvable Morphologies , 2018, PPSN.

[4]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Qi Hao,et al.  A Bayesian Approach for Targets Localization Using Binary Sensor Networks , 2015, 2015 8th International Symposium on Computational Intelligence and Design (ISCID).

[6]  Juan I. Nieto,et al.  A mutual information approach to automatic calibration of camera and lidar in natural environments , 2012 .

[7]  Qi Hao,et al.  Development of a Smart Floor for Target Localization with Bayesian Binary Sensing , 2017, 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA).

[8]  Kyungeun Cho,et al.  Calibration between Color Camera and 3D LIDAR Instruments with a Polygonal Planar Board , 2014, Sensors.

[9]  Nick Schneider,et al.  RegNet: Multimodal sensor registration using deep neural networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[10]  Qi Hao,et al.  Convolution neutral network enhanced binary sensor network for human activity recognition , 2016, 2016 IEEE SENSORS.

[11]  Andreas Geiger,et al.  Automatic camera and range sensor calibration using a single shot , 2012, 2012 IEEE International Conference on Robotics and Automation.

[12]  Ricardo Omar Chávez García,et al.  Multiple Sensor Fusion and Classification for Moving Object Detection and Tracking , 2016, IEEE Transactions on Intelligent Transportation Systems.

[13]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[14]  Takeo Kanade,et al.  Extrinsic calibration of a single line scanning lidar and a camera , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Silvio Savarese,et al.  Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information , 2012, AAAI.

[16]  Sebastian Thrun,et al.  Automatic Online Calibration of Cameras and Lasers , 2013, Robotics: Science and Systems.

[17]  Roland Siegwart,et al.  Self-supervised calibration for robotic systems , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[18]  Juan I. Nieto,et al.  Motion-based calibration of multimodal sensor arrays , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Vincent Frémont,et al.  Extrinsic calibration between a multi-layer lidar and a camera , 2008, 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[20]  Juan I. Nieto,et al.  Motion-Based Calibration of Multimodal Sensor Extrinsics and Timing Offset Estimation , 2016, IEEE Transactions on Robotics.

[21]  Quoc V. Le,et al.  Joint calibration of multiple sensors , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Takeshi Oishi,et al.  LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  A. E. Eiben,et al.  Real-Time Robot Vision on Low-Performance Computing Hardware , 2018, 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV).

[24]  Ganesh Iyer,et al.  CalibNet: Self-Supervised Extrinsic Calibration using 3D Spatial Transformer Networks , 2018, ArXiv.

[25]  B. V. K. Vijaya Kumar,et al.  A multi-sensor fusion system for moving object detection and tracking in urban driving environments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).