Importance Weighted Gaussian Process Regression for Transferable Driver Behaviour Learning in the Lane Change Scenario

Due to advantages of handling problems with nonlinearity and uncertainty, Gaussian process regression (GPR) has been widely used in the area of driver behaviour modelling. However, traditional GPR lacks the ability of transferring knowledge from one driver to another, which limits the generalisation ability of GPR, especially when sufficient data for driver behaviour modelling are not available. To solve this limitation, in this paper, a novel GPR model, Importance Weighted Gaussian Process Regression (IWGPR) is proposed. The importance weight (IW) represents the probabilistic density ratio between two drivers and the unconstrained least-squares importance fitting (ULSIF) is applied to calculate IW. Meanwhile, an IW-based model selection (IWMS) method is proposed to help the model select optimal parameters. Using IWGPR, sufficient historical data collected from one driver can be used to model another driver with insufficient data, and thus improve the generalisation ability of GPR. To verify the proposed algorithm, a toy regression problem is used to illustrate the working mechanism of IWGPR. With simulated and naturalistic driving data, three experiments for driver behaviour modelling in the lane change scenario, are designed and carried out. Experimental results indicate that IWGPR performs better than GPR when sufficient data are not provided by the new driver, which proves the generalisation ability of IWGPR. Meanwhile, the comparative study between different transferable driver behaviour learning methods is detailed and analysed.

[1]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[2]  Jin Gao,et al.  Transfer Learning Based Visual Tracking with Gaussian Processes Regression , 2014, ECCV.

[3]  Junqiang Xi,et al.  Learning Driver-Specific Behavior for Overtaking: A Combined Learning Framework , 2018, IEEE Transactions on Vehicular Technology.

[4]  Yunfeng Ai,et al.  Driver Lane Change Intention Inference for Intelligent Vehicles: Framework, Survey, and Challenges , 2019, IEEE Transactions on Vehicular Technology.

[5]  Benjamin Rosman,et al.  Knowledge transfer for learning robot models via Local Procrustes Analysis , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[6]  Fei-Yue Wang,et al.  Capturing Car-Following Behaviors by Deep Learning , 2018, IEEE Transactions on Intelligent Transportation Systems.

[7]  Jianwei Gong,et al.  Transferable Driver Behavior Learning via Distribution Adaption in the Lane Change Scenario* , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[8]  Qiang Yang,et al.  Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10) Adaptive Transfer Learning , 2022 .

[9]  Dongpu Cao,et al.  A Personalized Behavior Learning System for Human-Like Longitudinal Speed Control of Autonomous Vehicles , 2019, Sensors.

[10]  Dongpu Cao,et al.  Transfer Learning for Driver Model Adaptation in Lane-Changing Scenarios Using Manifold Alignment , 2020, IEEE Transactions on Intelligent Transportation Systems.

[11]  Bin Shuai,et al.  Dual-loop online intelligent programming for driver-oriented predict energy management of plug-in hybrid electric vehicles , 2019, Applied Energy.

[12]  Luis Miguel Bergasa,et al.  Need data for driver behaviour analysis? Presenting the public UAH-DriveSet , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[13]  Gang Wang,et al.  Development and Evaluation of Two Learning-Based Personalized Driver Models for Pure Pursuit Path-Tracking Behaviors , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[14]  Masashi Sugiyama,et al.  Importance-weighted least-squares probabilistic classifier for covariate shift adaptation with application to human activity recognition , 2012, Neurocomputing.

[15]  Motoaki Kawanabe,et al.  Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , 2007, NIPS.

[16]  Dongpu Cao,et al.  Driver Activity Recognition for Intelligent Vehicles: A Deep Learning Approach , 2019, IEEE Transactions on Vehicular Technology.

[17]  Samuel Kaski,et al.  Focused multi-task learning in a Gaussian process framework , 2012, Machine Learning.

[18]  Melih Kandemir,et al.  Asymmetric Transfer Learning with Deep Gaussian Processes , 2015, ICML.

[19]  Mykel J. Kochenderfer,et al.  Analysis of Recurrent Neural Networks for Probabilistic Modeling of Driver Behavior , 2017, IEEE Transactions on Intelligent Transportation Systems.

[20]  Sarfraz Khurshid,et al.  DeepRoad: GAN-Based Metamorphic Testing and Input Validation Framework for Autonomous Driving Systems , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[21]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[22]  Huijing Zhao,et al.  On-Road Vehicle Trajectory Collection and Scene-Based Lane Change Analysis: Part II , 2017, IEEE Trans. Intell. Transp. Syst..

[23]  Eric P. Xing,et al.  Real-to-Virtual Domain Unification for End-to-End Autonomous Driving , 2018, ECCV.

[24]  Junqiang Xi,et al.  Learning and Inferring a Driver's Braking Action in Car-Following Scenarios , 2018, IEEE Transactions on Vehicular Technology.

[25]  Efstathios Velenis,et al.  An ensemble deep learning approach for driver lane change intention inference , 2020, Transportation Research Part C: Emerging Technologies.

[26]  Hui Zhang,et al.  Active Steering Actuator Fault Detection for an Automatically-Steered Electric Ground Vehicle , 2017, IEEE Transactions on Vehicular Technology.

[27]  Barbara Plank,et al.  Learning to select data for transfer learning with Bayesian Optimization , 2017, EMNLP.

[28]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[29]  Masayoshi Tomizuka,et al.  Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving , 2019, IEEE Transactions on Intelligent Transportation Systems.

[30]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[31]  Huijing Zhao,et al.  On-Road Vehicle Trajectory Collection and Scene-Based Lane Change Analysis: Part I , 2017, IEEE Transactions on Intelligent Transportation Systems.

[32]  Hong Liu,et al.  End-to-End Driving Activities and Secondary Tasks Recognition Using Deep Convolutional Neural Network and Transfer Learning , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[33]  Dongpu Cao,et al.  Virtual-to-Real Knowledge Transfer for Driving Behavior Recognition: Framework and a Case Study , 2019, IEEE Transactions on Vehicular Technology.

[34]  Takafumi Kanamori,et al.  A Least-squares Approach to Direct Importance Estimation , 2009, J. Mach. Learn. Res..

[35]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[36]  Luis Miguel Bergasa,et al.  A Real-Time Multi-scale Vehicle Detection and Tracking Approach for Smartphones , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[37]  Sinno Jialin Pan,et al.  Knowledge Transfer for Deep Reinforcement Learning with Hierarchical Experience Replay , 2017, AAAI.

[38]  Wenshuo Wang,et al.  Transfer Learning for Driver Model Adaptation via Modified Local Procrustes Analysis , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[39]  Jianwei Gong,et al.  A Comparative Study on Transferable Driver Behavior Learning Methods in the Lane-Changing Scenario , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[40]  Jan Peters,et al.  Alignment-based transfer learning for robot models , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[41]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.