SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation

Learning models on one labeled dataset that generalize well on another domain is a difficult task, as several shifts might happen between the data domains. This is notably the case for lidar data, for which models can exhibit large performance discrepancies due for instance to different lidar patterns or changes in acquisition conditions. This paper addresses the corresponding Unsupervised Domain Adaptation (UDA) task for semantic segmentation. To mitigate this problem, we introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data. As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data. This novel strategy differs from classical minimization of statistical divergences or lidar-specific state-of-the-art domain adaptation techniques. Our experiments demonstrate that our method achieves a better performance than the current state of the art in synthetic-to-real and real-to-real scenarios.

[1]  Hongsheng Li,et al.  ST3D++: Denoised Self-Training for Unsupervised Domain Adaptation on 3D Object Detection , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Raoul de Charette,et al.  Cross-Modal Learning for Domain Adaptation in 3D Semantic Segmentation , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  G. Puy,et al.  ALSO: Automotive Lidar Self-Supervision by Occupancy Estimation , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yuexin Ma,et al.  CL3D: Unsupervised Domain Adaptation for Cross-LiDAR 3D Detection , 2022, AAAI.

[5]  Fabio Galasso,et al.  CoSMix: Compositional Semantic Mix for Domain Adaptation in 3D LiDAR Segmentation , 2022, ECCV.

[6]  Mohan S. Kankanhalli,et al.  Self-Supervised Global-Local Structure Modeling for Point Cloud Domain Adaptation with Reliable Voted Pseudo Labels , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Haotian Tang,et al.  TorchSparse: Efficient Point Cloud Inference Engine , 2022, MLSys.

[8]  Jiwen Lu,et al.  LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection , 2022, ECCV.

[9]  Renaud Marlet,et al.  Deep Surface Reconstruction from Point Clouds with Visibility Information , 2022, 2022 26th International Conference on Pattern Recognition (ICPR).

[10]  Renaud Marlet,et al.  POCO: Point Convolution for Surface Reconstruction , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  He Wang,et al.  Domain Adaptation on Point Clouds via Geometry-Aware Implicits , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  H. Bischof,et al.  The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  J. S. Berrio,et al.  See Eye to Eye: A Lidar-Agnostic 3D Detection Framework for Unsupervised Multi-Target Domain Adaptation , 2021, IEEE Robotics and Automation Letters.

[14]  Rohit Mohan,et al.  Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking , 2021, IEEE Robotics and Automation Letters.

[15]  Eduardo R. Corral-Soto,et al.  Unsupervised Domain Adaptation in LiDAR Semantic Segmentation with Self-Supervision and Gated Adapters , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[16]  Shijian Lu,et al.  Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation , 2021, AAAI.

[17]  Kilian Q. Weinberger,et al.  Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection in Self-Driving Cars , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[18]  Dariu M. Gavrila,et al.  Semantic Scene Completion Using Local Deep Implicit Functions on LiDAR Data , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Kevin Musgrave,et al.  Benchmarking Validation Methods for Unsupervised Domain Adaptation , 2022, ArXiv.

[20]  Y. Qiao,et al.  ADAS: A Simple Active-and-Adaptive Baseline for Cross-Domain 3D Semantic Segmentation , 2022, ArXiv.

[21]  Zhangyang Wang,et al.  Point Cloud Domain Adaptation via Masked Local 3D Structure Prediction , 2022, ECCV.

[22]  Venice Erin Liong,et al.  ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation , 2021, 2023 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Geoffrey A. Hollinger,et al.  Adversarial Training on Point Clouds for Sim-to-Real 3D Object Detection , 2021, IEEE Robotics and Automation Letters.

[24]  Kui Jia,et al.  Geometry-Aware Self-Training for Unsupervised Domain Adaptation on Object Point Clouds , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[25]  Charles R. Qi,et al.  SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Yulan Guo,et al.  Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[27]  Shijian Lu,et al.  Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Christoph B. Rist,et al.  A Survey on Deep Domain Adaptation for LiDAR Perception , 2021, 2021 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops).

[29]  Dong Xu,et al.  SRDAN: Scale-aware and Range-aware Domain Adaptation Network for Cross-dataset 3D Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  José Marcato Junior,et al.  Adversarial unsupervised domain adaptation for 3D semantic segmentation with multi-modal learning , 2021 .

[31]  Srikanth Saripalli,et al.  LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Carlos Guindel,et al.  Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View , 2021, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).

[33]  Xiaoyuan Luo,et al.  A Learnable Self-supervised Task for Unsupervised Domain Adaptation on Point Clouds , 2021, ArXiv.

[34]  Xiaojuan Qi,et al.  ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Nicolas Courty,et al.  Unbalanced minibatch Optimal Transport; applications to Domain Adaptation , 2021, ICML.

[36]  Ana Cristina Murillo,et al.  Domain Adaptation in LiDAR Semantic Segmentation by Aligning Class Distributions , 2020, ICINCO.

[37]  Trevor Darrell,et al.  ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework for LiDAR Point Cloud Segmentation , 2020, AAAI.

[38]  Thomas Funkhouser,et al.  Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Gal Chechik,et al.  Self-Supervised Learning for Domain Adaptation on Point Clouds , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[40]  Trevor Darrell,et al.  Tent: Fully Test-Time Adaptation by Entropy Minimization , 2021, ICLR.

[41]  C. Stachniss,et al.  Domain Transfer for Semantic Segmentation of LiDAR Data using Deep Neural Networks , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[42]  Dariu M. Gavrila,et al.  SCSSnet: Learning Spatially-Conditioned Scene Segmentation on LiDAR Point Clouds , 2020, 2020 IEEE Intelligent Vehicles Symposium (IV).

[43]  Cristiano Saltori,et al.  SF-UDA 3D : Source-Free Unsupervised Domain Adaptation for LiDAR-Based 3D Object Detection. , 2020 .

[44]  Song Han,et al.  Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution , 2020, ECCV.

[45]  Matthias Bethge,et al.  Improving robustness against common corruptions by covariate shift adaptation , 2020, NeurIPS.

[46]  D. Sculley,et al.  Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift , 2020, ArXiv.

[47]  Yan Wang,et al.  Train in Germany, Test in the USA: Making 3D Object Detectors Generalize , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Marc Pollefeys,et al.  Convolutional Occupancy Networks , 2020, ECCV.

[49]  Raoul de Charette,et al.  xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Diane J. Cook,et al.  A Survey of Unsupervised Deep Domain Adaptation , 2018, ACM Trans. Intell. Syst. Technol..

[51]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[52]  C.-C. Jay Kuo,et al.  PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representation , 2019, NeurIPS.

[53]  Dariu M. Gavrila,et al.  Cross-Sensor Deep Domain Adaptation for LiDAR Detection and Segmentation , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[54]  Silvio Savarese,et al.  4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Cyrill Stachniss,et al.  SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[56]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Sebastian Nowozin,et al.  Occupancy Networks: Learning 3D Reconstruction in Function Space , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Hao Zhang,et al.  Learning Implicit Fields for Generative Shape Modeling , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Patrick Pérez,et al.  ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Kurt Keutzer,et al.  SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[61]  Frank Hutter,et al.  Decoupled Weight Decay Regularization , 2017, ICLR.

[62]  Hongen Liao,et al.  Efficient Semantic Scene Completion Network with Spatial Group Convolution , 2018, ECCV.

[63]  Yang Zou,et al.  Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training , 2018, ArXiv.

[64]  Jiaying Liu,et al.  Adaptive Batch Normalization for practical domain adaptation , 2018, Pattern Recognit..

[65]  Nicolas Courty,et al.  DeepJDOT: Deep Joint distribution optimal transport for unsupervised domain adaptation , 2018, ECCV.

[66]  Chuang Gan,et al.  Unsupervised Domain Adaptation for 3D Keypoint Estimation via View Consistency , 2017, ECCV.

[67]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[68]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[69]  Michael I. Jordan,et al.  Conditional Adversarial Domain Adaptation , 2017, NeurIPS.

[70]  Luc Van Gool,et al.  Deep Domain Adaptation by Geodesic Distance Minimization , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[71]  Tatsuya Harada,et al.  Asymmetric Tri-training for Unsupervised Domain Adaptation , 2017, ICML.

[72]  Harri Valpola,et al.  Weight-averaged consistency targets improve semi-supervised deep learning results , 2017, ArXiv.

[73]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[75]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[76]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[77]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[78]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[79]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[80]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[81]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[82]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[83]  Daniel Marcu,et al.  Domain Adaptation for Statistical Classifiers , 2006, J. Artif. Intell. Res..