Energy-Efficient Autonomous Driving Using Cognitive Driver Behavioral Models and Reinforcement Learning

Autonomous driving technologies are expected to not only improve mobility and road safety but also bring energy e ciency benefits. In the foreseeable future, autonomous vehicles (AVs) will operate on roads shared with human-driven vehicles. To maintain safety and liveness while simultaneously minimizing energy consumption, the AV planning and decision-making process should account for interactions between the autonomous ego vehicle and surrounding human-driven vehicles. In this chapter, we describe a framework for developing energy-e cient autonomous driving policies on shared roads by exploiting human-driver behavior modeling based on cognitive hierarchy theory and reinforcement learning.

[1]  Zhijun Peng,et al.  Life Cycle Assessment (LCA) of BEV’s environmental benefits for meeting the challenge of ICExit (Internal Combustion Engine Exit) , 2021 .

[2]  Yi Lu Murphey,et al.  Intelligent Hybrid Vehicle Power Control—Part I: Machine Learning of Optimal Vehicle Power , 2012, IEEE Transactions on Vehicular Technology.

[3]  Anouck Girard,et al.  Coordinated Receding-Horizon Control of Battery Electric Vehicle Speed and Gearshift Using Relaxed Mixed-Integer Nonlinear Programming , 2021, IEEE Transactions on Control Systems Technology.

[4]  Christos G. Cassandras,et al.  Cooperative Time and Energy-Optimal Lane Change Maneuvers for Connected Automated Vehicles , 2022, IEEE Transactions on Intelligent Transportation Systems.

[5]  Victor Talpaert,et al.  Deep Reinforcement Learning for Autonomous Driving: A Survey , 2020, IEEE Transactions on Intelligent Transportation Systems.

[6]  Yu Yao,et al.  Game-Theoretic Modeling of Multi-Vehicle Interactions at Uncontrolled Intersections , 2019, IEEE Transactions on Intelligent Transportation Systems.

[7]  K. B. Wipke,et al.  ADVISOR 2.1: a user-friendly advanced powertrain simulation using a combined backward/forward approach , 1999 .

[8]  Ilya Kolmanovsky,et al.  Hierarchical MPC for Robust Eco-Cooling of Connected and Automated Vehicles and Its Application to Electric Vehicle Battery Thermal Management , 2021, IEEE Transactions on Control Systems Technology.

[9]  Georg Schildbach,et al.  A path planner for autonomous driving on highways using a human mimicry approach with Binary Decision Diagrams , 2015, 2015 European Control Conference (ECC).

[10]  H. Gamage,et al.  Reinforcement Learning based Driving Speed Control for Two Vehicle Scenario , 2017 .

[11]  Andreas A. Malikopoulos,et al.  Decentralized Cooperative Merging of Platoons of Connected and Automated Vehicles at Highway On-Ramps , 2021, 2021 American Control Conference (ACC).

[12]  Reza Langari,et al.  A Stackelberg Game Theoretic Driver Model for Merging , 2013 .

[13]  Wei Liang,et al.  Power Smoothing Energy Management and Its Application to a Series Hybrid Powertrain , 2013, IEEE Transactions on Control Systems Technology.

[14]  Francesco Borrelli,et al.  Scenario model predictive control for lane change assistance on highways , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[15]  Francesco Borrelli,et al.  Control of Connected and Automated Vehicles: State of the Art and Future Challenges , 2018, Annu. Rev. Control..

[16]  Nan Li,et al.  Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems , 2016, IEEE Transactions on Control Systems Technology.

[17]  Chang Liu,et al.  Power management for Plug-in Hybrid Electric Vehicles using Reinforcement Learning with trip information , 2014, 2014 IEEE Transportation Electrification Conference and Expo (ITEC).

[18]  Reza Langari,et al.  Stackelberg Game Based Model of Highway Driving , 2012 .

[19]  Stefano Longo,et al.  Comparative analysis of forward-facing models vs backward-facing models in powertrain component sizing , 2013 .

[20]  Mortuza Ali,et al.  An efficient cooperative lane-changing algorithm for sensor- and communication-enabled automated vehicles , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[21]  Nan Li,et al.  Game-Theoretic Modeling of Traffic in Unsignalized Intersection Network for Autonomous Vehicle Control Verification and Validation , 2019, IEEE Transactions on Intelligent Transportation Systems.

[22]  Ardalan Vahidi,et al.  Energy saving potentials of connected and automated vehicles , 2018, Transportation Research Part C: Emerging Technologies.

[23]  Anouck Girard,et al.  Game Theory-Based Traffic Modeling for Calibration of Automated Driving Algorithms , 2018, Control Strategies for Advanced Driver Assistance Systems and Autonomous Driving Functions.

[24]  Richard Bowden,et al.  A Survey of Deep Learning Applications to Autonomous Vehicle Control , 2019, IEEE Transactions on Intelligent Transportation Systems.

[25]  Guillaume Brat,et al.  Predicting Pilot Behavior in Medium Scale Scenarios Using Game Theory and Reinforcement Learning , 2013 .

[26]  Paola Salomoni,et al.  On combining Big Data and machine learning to support eco-driving behaviours , 2019, Journal of Big Data.

[27]  Ilya V. Kolmanovsky,et al.  Recurrent neural network training for energy management of a mild Hybrid Electric Vehicle with an ultra-capacitor , 2009, 2009 IEEE Workshop on Computational Intelligence in Vehicles and Vehicular Systems.

[28]  Control Strategies for Advanced Driver Assistance Systems and Autonomous Driving Functions , 2019, Lecture Notes in Control and Information Sciences.

[29]  Teng Liu,et al.  Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects , 2019, IEEE Industrial Electronics Magazine.

[30]  Sorin Grigorescu,et al.  A Survey of Deep Learning Techniques for Autonomous Driving , 2020, J. Field Robotics.

[31]  Xiaosong Hu,et al.  Ensemble Reinforcement Learning-Based Supervisory Control of Hybrid Electric Vehicle for Fuel Economy Improvement , 2020, IEEE Transactions on Transportation Electrification.

[32]  Ragunathan Rajkumar,et al.  MultiCruise: Eco-Lane Selection Strategy with Eco-Cruise Control for Connected and Automated Vehicles , 2021, 2021 IEEE Intelligent Vehicles Symposium (IV).

[33]  Ardalan Vahidi,et al.  Receding Horizon Motion Planning for Automated Lane Change and Merge Using Monte Carlo Tree Search and Level-K Game Theory , 2020, 2020 American Control Conference (ACC).

[34]  Hao Wang,et al.  Combined Energy and Comfort Optimization of Air Conditioning System in Connected and Automated Vehicles , 2019, ArXiv.

[35]  Michael I. Jordan,et al.  Reinforcement Learning Algorithm for Partially Observable Markov Decision Problems , 1994, NIPS.

[36]  Peng Hao,et al.  Deep Learning–based Eco-driving System for Battery Electric Vehicles , 2019 .

[37]  Ilya V. Kolmanovsky,et al.  Emissions Modeling of a Light-Duty Diesel Engine for Model-Based Control Design Using Multi-Layer Perceptron Neural Networks , 2017 .

[38]  Lino Guzzella,et al.  Vehicle Propulsion Systems , 2013 .

[39]  Christos G. Cassandras,et al.  Eco-Driving of Autonomous Vehicles for Nonstop Crossing of Signalized Intersections , 2022, IEEE Transactions on Automation Science and Engineering.

[40]  Andreas A. Malikopoulos,et al.  Decentralized optimal coordination of connected and automated vehicles for multiple traffic scenarios , 2020, Autom..

[41]  Pierluigi Pisu,et al.  Closed‐loop hierarchical control strategies for connected and autonomous hybrid electric vehicles with random errors , 2018, IET Intelligent Transport Systems.

[42]  Christos G. Cassandras,et al.  Combined Eco-Routing and Power-Train Control of Plug-In Hybrid Electric Vehicles in Transportation Networks , 2020, IEEE Transactions on Intelligent Transportation Systems.

[43]  Mahdi Shahbakhti,et al.  Modeling of Thermal Dynamics of a Connected Hybrid Electric Vehicle for Integrated HVAC and Powertrain Optimal Operation , 2019, Volume 2: Modeling and Control of Engine and Aftertreatment Systems; Modeling and Control of IC Engines and Aftertreatment Systems; Modeling and Validation; Motion Planning and Tracking Control; Multi-Agent and Networked Systems; Renewable and Smart Energ.

[44]  Ilya Kolmanovsky,et al.  Stochastic Fuel Efficient Optimal Control of Vehicle Speed , 2014 .