Panoramic Image Mosaics Assisted by LiDAR in Vehicle System

Panoramic sensing system has attracted great attention in intelligent vehicle because of the wide range of perception areas. In order to make the spatial direction of each pixel more precise in panoramic image, we propose a panoramic image mosaics assisted by LiDAR. Firstly, a fusion system consisting of LiDAR and panoramic camera is built, and the related calibration equipment is elaborately designed for the system considering the sparse point cloud from 16-line LiDAR. Secondly, the center point of calibration plate is utilized to construct the geometry constraint for the extrinsic calibration of dual fisheye cameras, and the rotation matrix of calibration parameters makes the alignment of dual fisheye images. Thirdly, in order to offset calibration deviation and the distance between optical centers, the cos(x) smoothing function is introduced to eliminate the splicing marks in final panoramic image. Experimental results show that the proposed method can effectively be used in panoramic image mosaics, and the spatial direction of each pixel in the panoramic image is accurate.

[1]  Francesc Moreno-Noguer,et al.  Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Dushyant Vaghela,et al.  A Review of Image Mosaicing Techniques , 2014, ArXiv.

[4]  Narendra Ahuja,et al.  An Omnidirectional Stereo Vision System Using a Single Camera , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[5]  Madhukar Budagavi,et al.  Dual-fisheye lens stitching for 360-degree imaging , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Juho Kannala,et al.  A generic camera calibration method for fish-eye lenses , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[7]  Juraj Pervsi'c.  Calibration of Heterogeneous Sensor Systems , 2018 .

[8]  F. A. Heuvel,et al.  CALIBRATION OF FISHEYE CAMERA SYSTEMS AND THE REDUCTION OF CHROMATIC ABERRATION , 2006 .

[9]  Jun Wang,et al.  Surrounding Vehicle Detection Using an FPGA Panoramic Camera and Deep CNNs , 2020, IEEE Transactions on Intelligent Transportation Systems.

[10]  Yael Pritch,et al.  Megastereo: Constructing High-Resolution Stereo Panoramas , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Jian Bai,et al.  Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).