Distributed predictive cruise control based on reinforcement learning and validation on microscopic traffic simulation

This study proposes a novel distributed predictive cruise control (PCC) algorithm based on reinforcement learning. The algorithm aims at reducing idle time and maintaining an adjustable speed depending on the traffic signals. The effectiveness of the proposed approach has been validated through Paramics microscopic traffic simulations by proposing a scenario in Statesboro, Georgia. For different traffic demands, the travel time and fuel consumption rate of vehicles are compared between non-PCC and PCC algorithms. Microscopic traffic simulation results demonstrate that the proposed PCC algorithm will reduce the fuel consumption rate by 4.24% and decrease the average travel time by 3.78%.

[1]  Wei Wang,et al.  Reinforcement Learning-Based Variable Speed Limit Control Strategy to Reduce Traffic Congestion at Freeway Recurrent Bottlenecks , 2017, IEEE Transactions on Intelligent Transportation Systems.

[2]  Fenghua Zhu,et al.  Parallel Transportation Management and Control System and Its Applications in Building Smart Cities , 2016, IEEE Transactions on Intelligent Transportation Systems.

[3]  Zhong-Ping Jiang,et al.  Nonlinear and Adaptive Suboptimal Control of Connected Vehicles: A Global Adaptive Dynamic Programming Approach , 2017, J. Intell. Robotic Syst..

[4]  Dong Ngoduy Application of gas-kinetic theory to modelling mixed traffic of manual and ACC vehicles , 2012 .

[5]  Fei Li,et al.  Fuzzy Observer-Based Prescribed Performance Control of Vehicle Roll Behavior via Controllable Damper , 2019, IEEE Access.

[6]  Daniel Dajun Zeng,et al.  A Bibliographic Analysis of the IEEE Transactions on Intelligent Transportation Systems Literature , 2010, IEEE Trans. Intell. Transp. Syst..

[7]  Jianqiang Wang,et al.  Model Predictive Multi-Objective Vehicular Adaptive Cruise Control , 2011, IEEE Transactions on Control Systems Technology.

[8]  Jing Na,et al.  Online H∞ control for completely unknown nonlinear systems via an identifier–critic-based ADP structure , 2019, Int. J. Control.

[9]  Han Zhang,et al.  Urban Transit Coordination Using an Artificial Transportation System , 2011, IEEE Transactions on Intelligent Transportation Systems.

[10]  Fei-Yue Wang Driving into the Future with ITS , 2006, IEEE Intelligent Systems.

[11]  Dongbin Zhao,et al.  Adaptive Optimal Control of Heterogeneous CACC System With Uncertain Dynamics , 2019, IEEE Transactions on Control Systems Technology.

[12]  Jun Tan,et al.  Parameterized Batch Reinforcement Learning for Longitudinal Control of Autonomous Land Vehicles , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Hesham Rakha,et al.  Virginia Tech Comprehensive Power-Based Fuel Consumption Model: Model Development and Testing , 2011 .

[14]  Loren Bloomberg,et al.  Comparison of VISSIM and CORSIM Traffic Simulation Models on a Congested Network , 2000 .

[15]  Nathan van de Wouw,et al.  Controller Synthesis for String Stability of Vehicle Platoons , 2014, IEEE Transactions on Intelligent Transportation Systems.

[16]  Jing Na,et al.  Online optimal solutions for multi-player nonzero-sum game with completely unknown dynamics , 2017, Neurocomputing.

[17]  Yechen Qin,et al.  Online Energy Management for Multimode Plug-In Hybrid Electric Vehicles , 2019, IEEE Transactions on Industrial Informatics.

[18]  Petros A. Ioannou,et al.  Autonomous intelligent cruise control , 1993 .

[19]  Cheng Chen,et al.  Research Collaboration and ITS Topic Evolution: 10 Years at T-ITS , 2010, IEEE Transactions on Intelligent Transportation Systems.

[20]  Stefan Parkvall,et al.  5G wireless access: requirements and realization , 2014, IEEE Communications Magazine.

[21]  Kaan Ozbay,et al.  Reinforcement-Learning-Based Cooperative Adaptive Cruise Control of Buses in the Lincoln Tunnel Corridor with Time-Varying Topology , 2019, IEEE Transactions on Intelligent Transportation Systems.

[22]  Weihao Hu,et al.  A Heuristic Planning Reinforcement Learning-Based Energy Management for Power-Split Plug-in Hybrid Electric Vehicles , 2019, IEEE Transactions on Industrial Informatics.

[23]  Nan Li,et al.  Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems , 2016, IEEE Transactions on Control Systems Technology.

[24]  Zhong-Ping Jiang,et al.  Predictive cruise control of connected and autonomous vehicles via reinforcement learning , 2019, IET Control Theory & Applications.

[25]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[26]  Bogdan Trasnea,et al.  NeuroTrajectory: A Neuroevolutionary Approach to Local State Trajectory Learning for Autonomous Vehicles , 2019, IEEE Robotics and Automation Letters.

[27]  Song Guo,et al.  CACC: A context-aware congestion control approach in smartphone networks , 2014, IEEE Communications Magazine.

[28]  Ardalan Vahidi,et al.  Predictive Cruise Control: Utilizing Upcoming Traffic Signal Information for Improving Fuel Economy and Reducing Trip Time , 2011, IEEE Transactions on Control Systems Technology.

[29]  Zhong-Ping Jiang,et al.  Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems , 2016, IEEE Transactions on Automatic Control.

[30]  Ulf Holmberg,et al.  A Modular CACC System Integration and Design , 2012, IEEE Transactions on Intelligent Transportation Systems.

[31]  Ge Guo,et al.  Sampled-Data Cooperative Adaptive Cruise Control of Vehicles With Sensor Failures , 2014, IEEE Transactions on Intelligent Transportation Systems.

[32]  Bo Wang,et al.  Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning , 2018, Energy.

[33]  Hesham A Rakha,et al.  Vehicle dynamics model for predicting maximum truck acceleration levels , 2001 .

[34]  Nathan van de Wouw,et al.  Cooperative Adaptive Cruise Control: Network-Aware Analysis of String Stability , 2014, IEEE Transactions on Intelligent Transportation Systems.

[35]  Baher Abdulhai,et al.  Multiagent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC): Methodology and Large-Scale Application on Downtown Toronto , 2013, IEEE Transactions on Intelligent Transportation Systems.