Direct Visual Localisation and Calibration for Road Vehicles in Changing City Environments

This paper presents a large-scale evaluation of a visual localisation method in a challenging city environment. Our system makes use of a map built by combining data from LIDAR and cameras mounted on a survey vehicle to build a dense appearance prior of the environment. We then localise by minimising the normalised information distance (NID) between a live camera image and an image generated from our prior. The use of NID produces a localiser that is robust to significant changes in scene appearance. Furthermore, NID can be used to compare images across different modalities, allowing us to use the same system to determine the extrinsic calibration between LIDAR and camera on the survey vehicle. We evaluate our system with a large-scale experiment consisting of over 450,000 camera frames collected over 110km of driving over a period of six months, and demonstrate reliable localisation even in the presence of illumination change, snow and seasonal effects.

[1]  Paul Newman,et al.  LAPS - localisation using appearance of prior structure: 6-DoF monocular camera localisation using prior pointclouds , 2012, 2012 IEEE International Conference on Robotics and Automation.

[2]  Sebastian Thrun,et al.  Robust vehicle localization in urban environments using probabilistic maps , 2010, 2010 IEEE International Conference on Robotics and Automation.

[3]  Paul Newman,et al.  FARLAP: Fast robust localisation using appearance priors , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[4]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  Paul Newman,et al.  TICSync: Knowing when things happened , 2011, 2011 IEEE International Conference on Robotics and Automation.

[6]  Paul Suetens,et al.  Medical image registration using mutual information , 2003, Proc. IEEE.

[7]  Gordon Wyeth,et al.  Extending the dynamic range of robotic vision , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[8]  Gordon Wyeth,et al.  FAB-MAP + RatSLAM: Appearance-based SLAM for multiple times of day , 2010, 2010 IEEE International Conference on Robotics and Automation.

[9]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[10]  Timothy D. Barfoot,et al.  Visual teach and repeat for long-range rover autonomy , 2010 .

[11]  Bin Ma,et al.  The similarity metric , 2001, IEEE Transactions on Information Theory.

[12]  Henning Lategahn,et al.  How to learn an illumination robust image feature for place recognition , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[13]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[14]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[15]  Paul Newman,et al.  FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance , 2008, Int. J. Robotics Res..

[16]  Ryan M. Eustice,et al.  Visual localization within LIDAR maps for automated urban driving , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Éric Marchand,et al.  Direct model based visual tracking and pose estimation using mutual information , 2014, Image Vis. Comput..

[18]  Paul Newman,et al.  Road vehicle localization with 2D push-broom LIDAR and 3D priors , 2012, 2012 IEEE International Conference on Robotics and Automation.

[19]  Peter I. Corke,et al.  Cross-calibration of push-broom 2D LIDARs and cameras in natural scenes , 2013, 2013 IEEE International Conference on Robotics and Automation.

[20]  Silvio Savarese,et al.  Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual Information , 2015, J. Field Robotics.

[21]  Julius Ziegler,et al.  Video based localization for Bertha , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[22]  Gordon Wyeth,et al.  Robust outdoor visual localization using a three‐dimensional‐edge map , 2009, J. Field Robotics.

[23]  Christian Früh,et al.  Google Street View: Capturing the World at Street Level , 2010, Computer.

[24]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Paul Newman,et al.  LAPS-II: 6-DoF day and night visual localisation with prior 3D structure for autonomous road vehicles , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[26]  Achim J. Lilienthal,et al.  SIFT, SURF and Seasons: Long-term Outdoor Localization Using Local Features , 2007, EMCR.

[27]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[28]  Paul Newman,et al.  Appearance-only SLAM at large scale with FAB-MAP 2.0 , 2011, Int. J. Robotics Res..

[29]  Paul Newman,et al.  Robust Direct Visual Localisation using Normalised Information Distance , 2015, BMVC.