Online Extrinsic Camera Calibration for Temporally Consistent IPM Using Lane Boundary Observations with a Lane Width Prior

In this paper, we propose a method for online extrinsic camera calibration, i.e., estimating pitch, yaw, roll angles and camera height from road surface in sequential driving scene images. The proposed method estimates the extrinsic camera parameters in two steps: 1) pitch and yaw angles are estimated simultaneously using a vanishing point computed from a set of lane boundary observations, and then 2) roll angle and camera height are computed by minimizing difference between lane width observations and a lane width prior. The extrinsic camera parameters are sequentially updated using extended Kalman filtering (EKF) and are finally used to generate a temporally consistent bird-eye-view (BEV) image by inverse perspective mapping (IPM). We demonstrate the superiority of the proposed method in synthetic and real-world datasets.

[1]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[2]  Shiyu Song,et al.  Joint SFM and detection cues for monocular 3D localization in road scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Baoli Li,et al.  Traffic-Sign Detection and Classification in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Grantham K. H. Pang,et al.  Camera calibration from road lane markings , 2003 .

[6]  Mohamed Aly,et al.  Real time detection of lane markers in urban streets , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[7]  Jinyong Jeong,et al.  Adaptive Inverse Perspective Mapping for lane map generation with SLAM , 2016, 2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[8]  Kuk-Jin Yoon,et al.  Joint Estimation of Camera Orientation and Vanishing Points from an Image Sequence in a Non-Manhattan World , 2019, International Journal of Computer Vision.

[9]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  Keechoo Choi,et al.  Generalized camera calibration model for trapezoidal patterns on the road , 2013 .

[12]  J. Little,et al.  Inverse perspective mapping simplifies optical flow computation and obstacle detection , 2004, Biological Cybernetics.

[13]  Ralf Reulke,et al.  Self-Calibration System for the Orientation of a Vehicle Camera , 2006 .

[14]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[15]  Amnon Shashua,et al.  Vision-based ACC with a single camera: bounds on range and range rate accuracy , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[16]  Bin Fang,et al.  Robust inverse perspective mapping based on vanishing point , 2014, Proceedings 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[17]  Shiyu Song,et al.  Robust Scale Estimation in Real-Time Monocular SFM for Autonomous Driving , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Ioana Maria Chira,et al.  Real-time detection of road markings for driving assistance applications , 2010, The 2010 International Conference on Computer Engineering & Systems.

[19]  Carsten Rother,et al.  A New Approach for Vanishing Point Detection in Architectural Environments , 2000, BMVC.

[20]  Bin Yang,et al.  Multi-Task Multi-Sensor Fusion for 3D Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Feng Xiao,et al.  Lane Detection Algorithm for Intelligent Vehicles in Complex Road Conditions and Dynamic Environments , 2019, Sensors.

[22]  Fei-Yue Wang,et al.  Research on lane-marking line based camera calibration , 2007, 2007 IEEE International Conference on Vehicular Electronics and Safety.

[23]  Sheng-Jyh Wang,et al.  A vision-based approach to extracting the tilt angle and altitude of a PTZ camera , 2006, Electronic Imaging.

[24]  Trevor Darrell,et al.  Joint Monocular 3D Vehicle Detection and Tracking , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).