Diff-Net: Image Feature Difference Based High-Definition Map Change Detection for Autonomous Driving

Up-to-date High-Definition (HD) maps are essential for self-driving cars. To achieve constantly updated HD maps, we present a deep neural network (DNN), Diff-Net, to detect changes in them. Compared to traditional methods based on object detectors, the essential design in our work is a parallel feature difference calculation structure that infers map changes by comparing features extracted from the camera and rasterized images. To generate these rasterized images, we project map elements onto images in the camera view, yielding meaningful map representations that can be consumed by a DNN accordingly. As we formulate the change detection task as an object detection problem, we leverage the anchor-based structure that predicts bounding boxes with different change status categories. To the best of our knowledge, the proposed method is the first end-to-end network that tackles the highdefinition map change detection task, yielding a single stage solution. Furthermore, rather than relying on single frame input, we introduce a spatio-temporal fusion module that fuses features from history frames into the current, thus improving the overall performance. Finally, we comprehensively validate our method’s effectiveness using freshly collected datasets. Results demonstrate that our Diff-Net achieves better performance than the baseline methods and is ready to be integrated into a map production pipeline maintaining an up-to-date HD map.

[1]  Marc Pollefeys,et al.  Geometric Change Detection in Urban Environments Using Images , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Takayuki Okatani,et al.  Change Detection from a Street Image Pair using CNN Features and Superpixel Segmentation , 2015, BMVC.

[3]  Joseph L. Mundy,et al.  Image-Based 4-d Reconstruction Using 3-d Change Detection , 2014, ECCV.

[4]  Wolfram Burgard,et al.  Driving Through Ghosts: Behavioral Cloning with False Positives , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Sergio Casas,et al.  IntentNet: Learning to Predict Intention from Raw Sensor Data , 2018, CoRL.

[6]  David B. Cooper,et al.  Using 3D Line Segments for Robust and Efficient Change Detection from Multiple Noisy Images , 2008, ECCV.

[7]  Yu Liu,et al.  Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change Detection , 2018, ArXiv.

[8]  Haifeng Li,et al.  Hierarchical Paired Channel Fusion Network for Street Scene Change Detection , 2021, IEEE Transactions on Image Processing.

[9]  Marc Pollefeys,et al.  Image based detection of geometric changes in urban environments , 2011, 2011 International Conference on Computer Vision.

[10]  Menglong Yan,et al.  Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[11]  Luc Van Gool,et al.  Learning Accurate and Human-Like Driving using Semantic Maps and Attention , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Ken Sakurada,et al.  Weakly Supervised Silhouette-based Semantic Scene Change Detection , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Jiaojiao Tian,et al.  3D change detection – Approaches and applications , 2016 .

[14]  Cyrill Stachniss,et al.  Fast Image-Based Geometric Change Detection Given a 3D Model , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Wolfram Burgard,et al.  HD Map Change Detection with a Boosted Particle Filter , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[16]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[17]  Benjamin Sapp,et al.  MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction , 2019, CoRL.

[18]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[19]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[22]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  A. Gruen,et al.  3D change detection at street level using mobile laser scanning point clouds and terrestrial images , 2014 .

[24]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[25]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[26]  Richard Szeliski,et al.  Building Rome in a day , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  Larry S. Davis,et al.  Soft-NMS — Improving Object Detection with One Line of Code , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Jiwon Kim,et al.  HD Map Change Detection with Cross-Domain Deep Metric Learning , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Henggang Cui,et al.  Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving , 2018, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[30]  Takayuki Okatani,et al.  Detecting Changes in 3D Structure of a Scene from Multi-view Images Captured by a Vehicle-Mounted Camera , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Hei Law,et al.  CornerNet: Detecting Objects as Paired Keypoints , 2018, ECCV.

[32]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Silvio Savarese,et al.  Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Mayank Bansal,et al.  ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst , 2018, Robotics: Science and Systems.

[35]  Carlos Hernandez,et al.  Multi-View Stereo: A Tutorial , 2015, Found. Trends Comput. Graph. Vis..

[36]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[37]  Marc Pollefeys,et al.  DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Germán Ros,et al.  Street-view change detection with deconvolutional networks , 2016, Autonomous Robots.

[39]  Joseph L. Mundy,et al.  Change Detection in a 3-d World , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Badrinath Roysam,et al.  Image change detection algorithms: a systematic survey , 2005, IEEE Transactions on Image Processing.

[41]  Silvio Savarese,et al.  Monitoring changes of 3D building elements from unordered photo collections , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).