Look Around You: Sequence-based Radar Place Recognition with Learned Rotational Invariance

This paper details an application which yields significant improvements to the adeptness of place recognition with Frequency-Modulated Continuous-Wave scanning, 360-degrees field of view radar - a commercially promising sensor poised for exploitation in mobile autonomy. We show how a rotationally-invariant metric embedding for radar scans can be integrated into sequence-based trajectory matching systems typically applied to videos taken by visual sensors. Due to the complete horizontal field of view inherent to the radar scan formation process, we show how this off-the-shelf sequence-based trajectory matching system can be manipulated to detect place matches when the vehicle is travelling down a previously visited stretch of road in the opposite direction. We demonstrate the efficacy of the approach on 26 km of challenging urban driving taken from the largest radar-focused urban autonomy dataset released to date - showing a boost of 30 % in recall at high levels of precision over a nearest neighbour approach.

[1]  Paul Newman,et al.  Radar-only ego-motion estimation in difficult settings via graph matching , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[2]  W. Marsden.  I and J , 2012 .

[3]  Jinyong Jeong,et al.  MulRan: Multimodal Range Dataset for Urban Place Recognition , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Howie Choset,et al.  Principles of Robot Motion: Theory, Algorithms, and Implementation ERRATA!!!! 1 , 2007 .

[5]  Walter R. Fried.  HISTORY OF DOPPLER RADAR NAVIGATION. , 1993 .

[6]  Michael J. Korenberg,et al.  Leveraging FMCW-Radar for Autonomous Positioning Systems: Methodology and Application in Downtown Toronto , 2019, Proceedings of the 32nd International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2019).

[7]  Richard Zhang,et al.  Making Convolutional Networks Shift-Invariant Again , 2019, ICML.

[8]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[9]  Nils Appenrodt,et al.  "Automotive radar the key technology for autonomous driving: From detection and ranging to environmental understanding" , 2016, 2016 IEEE Radar Conference (RadarConf).

[10]  Gordon Wyeth,et al.  SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights , 2012, 2012 IEEE International Conference on Robotics and Automation.

[11]  David M. Bevly,et al.  Radar probabilistic data association filter/DGPS fusion for target selection and relative range determination for a ground vehicle convoy , 2019 .

[12]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[13]  Matthew Gadd,et al.  Kidnapped Radar: Topological Radar Localisation using Rotationally-Invariant Metric Learning , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Weiliang Xu,et al.  Synchronous Adversarial Feature Learning for LiDAR based Loop Closure Detection , 2018, 2018 Annual American Control Conference (ACC).

[15]  Cyrill Stachniss,et al.  Effective Visual Place Recognition Using Multi-Sequence Maps , 2019, IEEE Robotics and Automation Letters.

[16]  Peter I. Corke,et al.  Automatic image scaling for place recognition in changing environments , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Brett Browning,et al.  Visual place recognition using HMM sequence matching , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Paul Newman,et al.  Precise Ego-Motion Estimation with Millimeter-Wave Radar Under Diverse and Challenging Conditions , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[20]  Masatoshi Okutomi,et al.  24/7 Place Recognition by View Synthesis , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Matthew Gadd,et al.  RSS-Net: Weakly-Supervised Multi-Class Semantic Segmentation with FMCW Radar , 2020, 2020 IEEE Intelligent Vehicles Symposium (IV).

[22]  Michael Milford,et al.  Addressing Challenging Place Recognition Tasks Using Generative Adversarial Networks , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Cyrill Stachniss,et al.  Lazy Data Association For Image Sequences Matching Under Substantial Appearance Changes , 2016, IEEE Robotics and Automation Letters.

[24]  Jian Bai,et al.  Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[25]  Paul Newman,et al.  Fast Radar Motion Estimation with a Learnt Focus of Attention using Weak Supervision , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[26]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Matthew Gadd,et al.  The Hulk: Design and Development of a Weather-Proof Vehicle for Long-Term Autonomy in Outdoor Environments , 2019, FSR.

[28]  Paul Newman,et al.  What Could Go Wrong? Introspective Radar Odometry in Challenging Environments , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[29]  Paul Newman,et al.  The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Ingmar Posner,et al.  Masking by Moving: Learning Distraction-Free Radar Odometry from Pose Information , 2019, CoRL.

[31]  Niko Sünderhauf,et al.  Are We There Yet? Challenging SeqSLAM on a 3000 km Journey Across All Four Seasons , 2013 .

[32]  Ingmar Posner,et al.  Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Tomás Pajdla,et al.  NetVLAD: CNN Architecture for Weakly Supervised Place Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Xuejun Yang,et al.  CNN Feature boosted SeqSLAM for Real-Time Loop Closure Detection , 2017, ArXiv.

[35]  Min Sun,et al.  Omnidirectional CNN for Visual Place Recognition and Navigation , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Paul Newman,et al.  RSL-Net: Localising in Satellite Images From a Radar on the Ground , 2020, IEEE Robotics and Automation Letters.

[37]  Paul Newman,et al.  Probably Unknown: Deep Inverse Sensor Modelling Radar , 2018, 2019 International Conference on Robotics and Automation (ICRA).