LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry

This paper proposes a targetless and automatic camera-LiDAR calibration method. Our approach extends the hand-eye calibration framework to 2D-3D calibration. The scaled camera motions are accurately calculated using a sensor-fusion odometry method. We also clarify the suitable motions for our calibration method. Whereas other calibrations require the LiDAR reflectance data and an initial extrinsic parameter, the proposed method requires only the three-dimensional point cloud and the camera image. The effectiveness of the method is demonstrated in experiments using several sensor configurations in indoor and outdoor scenes. Our method achieved higher accuracy than comparable state-of-the-art methods.

[1]  Thomas B. Schön,et al.  Modeling and Calibration of Inertial and Vision Sensors , 2010, Int. J. Robotics Res..

[2]  Juan I. Nieto,et al.  A mutual information approach to automatic calibration of camera and lidar in natural environments , 2012 .

[3]  Adrien Bartoli,et al.  Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces , 2013, BMVC.

[4]  David Johnson,et al.  Multi‐Modal Sensor Calibration Using a Gradient Orientation Measure , 2015, J. Field Robotics.

[5]  Gaurav S. Sukhatme,et al.  Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration , 2011, Int. J. Robotics Res..

[6]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[7]  Juan I. Nieto,et al.  Motion-Based Calibration of Multimodal Sensor Extrinsics and Timing Offset Estimation , 2016, IEEE Transactions on Robotics.

[8]  In-So Kweon,et al.  Extrinsic calibration of a camera and a 2D laser without overlap , 2016, Robotics Auton. Syst..

[9]  Atsushi Nakazawa,et al.  Fast simultaneous alignment of multiple range images using index images , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  Jianya Gong,et al.  Line-Based Registration of Panoramic Images and LiDAR Point Clouds for Mobile Mapping , 2017, Sensors.

[12]  Roland Siegwart,et al.  A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation , 2011, CVPR 2011.

[13]  Didier Stricker,et al.  Structure from Motion using full spherical panoramic cameras , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[14]  Sebastian Thrun,et al.  Automatic Online Calibration of Cameras and Lasers , 2013, Robotics: Science and Systems.

[15]  I. Fassi,et al.  Hand to sensor calibration: A geometrical interpretation of the matrix equation AX = XB , 2005 .

[16]  Yiu Cheung Shiu,et al.  Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX=XB , 1989, IEEE Trans. Robotics Autom..

[17]  Katsushi Ikeuchi,et al.  Omnidirectional texturing based on robust 3D registration through Euclidean reconstruction from two spherical images , 2010, Comput. Vis. Image Underst..

[18]  In-So Kweon,et al.  Capturing Village-level Heritages with a Hand-held Camera-Laser Fusion Sensor , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[19]  Robert Pless,et al.  Extrinsic calibration of a camera and laser range finder (improves camera calibration) , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[20]  Takeshi Oishi,et al.  A New Flying Range Sensor: Aerial Scan in Omni-Directions , 2015, 2015 International Conference on 3D Vision.

[21]  Kiyoshi Irie,et al.  Target-less camera-LiDAR extrinsic calibration using a bagged dependence estimator , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[22]  James R. Bergen,et al.  Visual odometry for ground vehicle applications , 2006, J. Field Robotics.

[23]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[24]  Marc Pollefeys,et al.  CamOdoCal: Automatic intrinsic and extrinsic calibration of a rig with multiple generic cameras and odometry , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Vincent Frémont,et al.  Extrinsic calibration between a multi-layer lidar and a camera , 2008, 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[26]  Ji Zhang,et al.  A real-time method for depth enhanced visual odometry , 2017, Auton. Robots.

[27]  Silvio Savarese,et al.  Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual Information , 2015, J. Field Robotics.

[28]  Takeshi Oishi,et al.  A 3D Reconstruction with High Density and Accuracy Using Laser Profiler and Camera Fusion System on a Rover , 2016, 2016 Fourth International Conference on 3D Vision (3DV).