Assessing Domain Gap for Continual Domain Adaptation in Object Detection

To ensure reliable object detection in autonomous systems, the detector must be able to adapt to changes in appearance caused by environmental factors such as time of day, weather, and seasons. Continually adapting the detector to incorporate these changes is a promising solution, but it can be computationally costly. Our proposed approach is to selectively adapt the detector only when necessary, using new data that does not have the same distribution as the current training data. To this end, we investigate three popular metrics for domain gap evaluation and find that there is a correlation between the domain gap and detection accuracy. Therefore, we apply the domain gap as a criterion to decide when to adapt the detector. Our experiments show that our approach has the potential to improve the efficiency of the detector's operation in real-world scenarios, where environmental conditions change in a cyclical manner, without sacrificing the overall performance of the detector. Our code is publicly available at https://github.com/dadung/DGE-CDA.

[1]  J. Choo,et al.  TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation , 2023, ICLR.

[2]  Jindong Wang,et al.  Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals , 2021, IEEE Transactions on Knowledge and Data Engineering.

[3]  Vishal M. Patel,et al.  Unsupervised Domain Adaptation of Object Detectors: A Survey , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Yifan Sun,et al.  H2FA R-CNN: Holistic and Hierarchical Feature Alignment for Cross-domain Weakly Supervised Object Detection , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  L. Gool,et al.  Continual Test-Time Domain Adaptation , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Gustavo E. A. P. A. Batista,et al.  Update Compression for Deep Neural Networks on the Edge , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Bo Han,et al.  SEAR: Scaling Experiences in Multi-user Augmented Reality , 2022, IEEE Transactions on Visualization and Computer Graphics.

[8]  Andreas Zell,et al.  Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles , 2021, 2022 26th International Conference on Pattern Recognition (ICPR).

[9]  Bichen Wu,et al.  Cross-Domain Adaptive Teacher for Object Detection , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Anh-Dzung Doan,et al.  Autonomy and Perception for Space Mining , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[11]  Mengyuan Liu,et al.  Consistency-based Active Learning for Object Detection , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Tinne Tuytelaars,et al.  A Continual Learning Survey: Defying Forgetting in Classification Tasks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Markus Reischl,et al.  Night-to-Day: Online Image-to-Image Translation for Object Detection Within Autonomous Driving by Night , 2021, IEEE Transactions on Intelligent Vehicles.

[14]  Kui Fu,et al.  Informative and Consistent Correspondence Mining for Cross-Domain Weakly Supervised Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Xiangyang Ji,et al.  Multiple Instance Active Learning for Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Hyuk-Jae Lee,et al.  Active Learning for Deep Object Detection via Probabilistic Modeling , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Giovanni Maria Farinella,et al.  An unsupervised domain adaptation scheme for single-stage artwork recognition in cultural sites , 2020, Image Vis. Comput..

[18]  Jiayi Ma,et al.  DE-CycleGAN: An Object Enhancement Network for Weak Vehicle Detection in Satellite Images , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Trevor Darrell,et al.  Tent: Fully Test-Time Adaptation by Entropy Minimization , 2021, ICLR.

[20]  Tatiana Tommasi,et al.  One-Shot Unsupervised Cross-Domain Detection , 2020, ECCV.

[21]  Gong Cheng,et al.  High-Quality Proposals for Weakly Supervised Object Detection , 2020, IEEE Transactions on Image Processing.

[22]  Trevor Darrell,et al.  BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning , 2018, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Dongrui Fan,et al.  C-MIDN: Coupled Multiple Instance Detection Network With Segmentation Guidance for Weakly Supervised Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[25]  Dimitris Samaras,et al.  Wasserstein GAN With Quadratic Transport Cost , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Rama Chellappa,et al.  Wasserstein Distance Based Domain Adaptation for Object Detection , 2019, ArXiv.

[27]  Tat-Jun Chin,et al.  Scalable Place Recognition Under Appearance Change for Autonomous Driving , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Nicu Sebe,et al.  Cross-Domain Car Detection Using Unsupervised Image-to-Image Translation: From Day to Night , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[29]  Liangliang Cao,et al.  Automatic Adaptation of Object Detectors to New Domains Using Self-Training , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Arash Vahdat,et al.  A Robust Learning Approach to Domain Adaptive Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[31]  Omar Y. Al-Jarrah,et al.  A Survey on 3D Object Detection Methods for Autonomous Driving Applications , 2019, IEEE Transactions on Intelligent Transportation Systems.

[32]  Joachim Denzler,et al.  Active Learning for Deep Object Detection , 2018, VISIGRAPP.

[33]  Jonas Adler,et al.  Banach Wasserstein GAN , 2018, NeurIPS.

[34]  Kiyoharu Aizawa,et al.  Cross-Domain Weakly-Supervised Object Detection Through Progressive Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Nicolas Courty,et al.  DeepJDOT: Deep Joint distribution optimal transport for unsupervised domain adaptation , 2018, ECCV.

[36]  Luc Van Gool,et al.  Domain Adaptive Faster R-CNN for Object Detection in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[37]  Vinay P. Namboodiri,et al.  Deep active learning for object detection , 2018, BMVC.

[38]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[39]  Luc Van Gool,et al.  Deep Domain Adaptation by Geodesic Distance Minimization , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[40]  Frank Keller,et al.  Training Object Class Detectors with Click Supervision , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Cordelia Schmid,et al.  Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Ivan Laptev,et al.  ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization , 2016, ECCV.

[45]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[46]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[48]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[49]  Andrea Vedaldi,et al.  Weakly Supervised Deep Detection Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Zoubin Ghahramani,et al.  Training generative neural networks via Maximum Mean Discrepancy optimization , 2015, UAI.

[53]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[54]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[55]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[56]  Zaïd Harchaoui,et al.  On learning to localize objects with minimal supervision , 2014, ICML.

[57]  T. Tuytelaars,et al.  Weakly Supervised Object Detection with Posterior Regularization , 2014 .

[58]  Winston Churchill,et al.  Experience-based navigation for long-term localisation , 2013, Int. J. Robotics Res..

[59]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[60]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[61]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[62]  C. Villani.  Optimal Transport: Old and New , 2008 .

[63]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .