Privacy-preserving and Uncertainty-aware Federated Trajectory Prediction for Connected Autonomous Vehicles

Deep learning is the method of choice for trajectory prediction for autonomous vehicles. Unfortunately, its data-hungry nature implicitly requires the availability of sufficiently rich and high-quality centralized datasets, which easily leads to privacy leakage. Besides, uncertainty-awareness becomes increasingly important for safety-crucial cyber physical systems whose prediction module heavily relies on machine learning tools. In this paper, we relax the data collection requirement and enhance uncertainty-awareness by using Federated Learning on Connected Autonomous Vehicles with an uncertainty-aware global objective. We name our algorithm as FLTP. We further introduce ALFLTP which boosts FLTP via using active learning techniques in adaptatively selecting participating clients. We consider both negative log-likelihood (NLL) and aleatoric uncertainty (AU) as client selection metrics. Experiments on Argoverse dataset show that FLTP significantly outperforms the model trained on local data. In addition, ALFLTP-AU converges faster in training regression loss and performs better in terms of NLL, minADE and MR than FLTP in most rounds, and has more stable round-wise performance than ALFLTP-NLL.

[1]  D. Cao,et al.  Prediction-Uncertainty-Aware Decision-Making for Autonomous Vehicles , 2022, IEEE Transactions on Intelligent Vehicles.

[2]  Caiwen Ding,et al.  Uncertainty Quantification of Collaborative Detection for Self-Driving , 2022, 2023 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Peng Li,et al.  Uncertainty Measured Active Client Selection for Federated Learning in Smart Grid , 2022, 2022 IEEE International Conference on Smart Internet of Things (SmartIoT).

[4]  K. Lu,et al.  HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  M. Pavone,et al.  Whose Track Is It Anyway? Improving Robustness to Tracking Errors with Affinity-based Trajectory Prediction , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Chunnan Wang,et al.  ATPFL: Automatic Trajectory Prediction Model Design under Federated Learning Framework , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Qi Alfred Chen,et al.  On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Alexandre Alahi,et al.  Vehicle trajectory prediction works, but not everywhere , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  V. Knoop,et al.  UQnet: Quantifying Uncertainty in Trajectory Prediction by a Non-Parametric and Generalizable Approach , 2022, SSRN Electronic Journal.

[10]  Ulrich Neumann,et al.  Collaborative Uncertainty in Multi-Agent Trajectory Forecasting , 2021, NeurIPS.

[11]  Hang Zhao,et al.  DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Jonathon Shlens,et al.  Scene Transformer: A unified architecture for predicting multiple agent trajectories , 2021, 2106.08417.

[13]  Kris Kitani,et al.  AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  Bolei Zhou,et al.  Multimodal Motion Prediction with Stacked Transformers , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jeannette Bohg,et al.  Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Carlo Pinciroli,et al.  Flow-FL: Data-Driven Federated Learning for Spatio-Temporal Predictions in Multi-Robot Systems , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2019, Found. Trends Mach. Learn..

[18]  Mingyu Fan,et al.  Robust Trajectory Prediction of Multiple Interacting Pedestrians via Incremental Active Learning , 2021, ICONIP.

[19]  Jianyu Wang,et al.  Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies , 2020, ArXiv.

[20]  Yi Shen,et al.  TNT: Target-driveN Trajectory Prediction , 2020, CoRL.

[21]  R. Urtasun,et al.  Learning Lane Graph Representations for Motion Forecasting , 2020, ECCV.

[22]  Dragomir Anguelov,et al.  VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  S. Shen,et al.  PiP: Planning-informed Trajectory Prediction for Autonomous Driving , 2020, ECCV.

[24]  Dongsuk Kum,et al.  SCALE-Net: Scalable Vehicle Trajectory Prediction Network under Random Number of Interacting Vehicles via Edge-enhanced Graph Convolutional Neural Network , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Freddy A. Boulton,et al.  CoverNet: Multimodal Behavior Prediction Using Trajectory Sets , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  J. Schneider,et al.  Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving , 2018, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[27]  Benjamin Sapp,et al.  MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction , 2019, CoRL.

[28]  Anuj Kumar,et al.  Active Federated Learning , 2019, ArXiv.

[29]  Mooi Choo Chuah,et al.  GRIP: Graph-based Interaction-aware Trajectory Prediction , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[30]  Mayank Bansal,et al.  ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst , 2018, Robotics: Science and Systems.

[31]  Henggang Cui,et al.  Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[32]  Mohan M. Trivedi,et al.  Convolutional Social Pooling for Vehicle Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[33]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[34]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[35]  Bran Selic,et al.  Understanding Uncertainty in Cyber-Physical Systems: A Conceptual Model , 2016, ECMFA.

[36]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[37]  Dorothy J. Glancy Privacy in Autonomous Vehicles , 2012 .