Data-Driven Adaptive Optimal Control of Connected Vehicles

In this paper, a data-driven non-model-based approach is proposed for the adaptive optimal control of a class of connected vehicles that is composed of $n$ human-driven vehicles only transmitting motional data and an autonomous vehicle in the tail receiving the broadcasted data from preceding vehicles by wireless vehicle-to-vehicle (V2V) communication devices. Considering the cases of range-limited V2V communication and input saturation, several optimal control problems are formulated to minimize the errors of distance and velocity and to optimize the fuel usage. By employing an adaptive dynamic programming technique, the optimal controllers are obtained without relying on the knowledge of system dynamics. The effectiveness of the proposed approaches is demonstrated via the online learning control of the connected vehicles in Paramics' traffic microsimulation.

[1]  Nathan van de Wouw,et al.  Lp String Stability of Cascaded Systems: Application to Vehicle Platooning , 2014, IEEE Transactions on Control Systems Technology.

[2]  Luigi del Re,et al.  A model predictive Cooperative Adaptive Cruise Control approach , 2013, 2013 American Control Conference.

[3]  Masayoshi Tomizuka,et al.  Smooth motion control of the adaptive cruise control system by a virtual lead vehicle , 2011 .

[4]  Ge Guo,et al.  Autonomous Platoon Control Allowing Range-Limited Sensors , 2012, IEEE Transactions on Vehicular Technology.

[5]  Petros A. Ioannou,et al.  Vehicle following control design for automated highway systems , 1997, 1997 IEEE 47th Vehicular Technology Conference. Technology in Motion.

[6]  D. C. Chin,et al.  Traffic-responsive signal timing for system-wide traffic control , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[7]  Patrizio Colaneri,et al.  Decentralized optimal control of a car platoon with guaranteed string stability , 2013, 2013 European Control Conference (ECC).

[8]  Martijn van Noort,et al.  Modelling cooperative driving in congestion shockwaves on a freeway network , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[9]  Srdjan S. Stankovic,et al.  Decentralized overlapping control of a platoon of vehicles , 2000, IEEE Trans. Control. Syst. Technol..

[10]  Kaan Ozbay,et al.  Evaluation of a Methodology for Scalable Dynamic Vehicular Ad Hoc Networks in a Well-Calibrated Test Bed for Vehicular Mobility , 2013 .

[11]  Kazutaka Adachi,et al.  Design of a headway distance control system for ACC , 2001 .

[12]  Petros A. Ioannou,et al.  Evaluation of ACC vehicles in mixed traffic: lane change effects and sensitivity analysis , 2005, IEEE Transactions on Intelligent Transportation Systems.

[13]  Jianqiang Wang,et al.  Model Predictive Multi-Objective Vehicular Adaptive Cruise Control , 2011, IEEE Transactions on Control Systems Technology.

[14]  Zhong-Ping Jiang,et al.  Adaptive optimal control of connected vehicles , 2015, 2015 10th International Workshop on Robot Motion and Control (RoMoCo).

[15]  Gábor Stépán,et al.  Traffic jams: dynamics and control , 2010, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[16]  M. Athans,et al.  On the optimal error regulation of a string of moving vehicles , 1966 .

[17]  Luke Ng,et al.  Reinforcement Learning of Dynamic Collaborative Driving , 2008 .

[18]  Qinglai Wei,et al.  Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming , 2012, Autom..

[19]  Nathan van de Wouw,et al.  Cooperative Adaptive Cruise Control: Network-Aware Analysis of String Stability , 2014, IEEE Transactions on Intelligent Transportation Systems.

[20]  Pravin Varaiya,et al.  Smart cars on smart roads: problems of control , 1991, IEEE Trans. Autom. Control..

[21]  L. C. Davis,et al.  Modifications of the optimal velocity traffic model to include delay due to driver reaction time , 2003 .

[22]  Giovanni Fiengo,et al.  On convergence and robustness of the Extended Cooperative Cruise Control , 2014, 53rd IEEE Conference on Decision and Control.

[23]  Vicente Milanés Montero,et al.  Cooperative Adaptive Cruise Control in Real Traffic Situations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[24]  Wolfram Mauser,et al.  On the Fundamental Diagram of Traffic Flow , 2006, SIAM J. Appl. Math..

[25]  Zhuo Wang,et al.  From model-based control to data-driven control: Survey, classification and perspective , 2013, Inf. Sci..

[26]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[27]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[28]  Christopher M. Clark,et al.  Reinforcement learning of dynamic collaborative driving Part II: lateral adaptive control , 2008 .

[29]  Zhong-Ping Jiang,et al.  Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems , 2016, IEEE Transactions on Automatic Control.

[30]  Miroslav Krstic,et al.  Inverse optimal stabilization of a rigid spacecraft , 1999, IEEE Trans. Autom. Control..

[31]  Gábor Orosz,et al.  Optimal control of connected vehicle systems , 2014, 53rd IEEE Conference on Decision and Control.

[32]  Andrew G. Barto,et al.  Adaptive linear quadratic control using policy iteration , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[33]  Warren B. Powell,et al.  Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics) , 2007 .

[34]  Charles Desjardins,et al.  Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach , 2011, IEEE Transactions on Intelligent Transportation Systems.

[35]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[36]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[37]  Gábor Orosz,et al.  Dynamics of connected vehicle systems with delayed acceleration feedback , 2014 .

[38]  Eduardo Sontag.  Input to State Stability: Basic Concepts and Results , 2008 .

[39]  Zongli Lin,et al.  Output regulation for linear systems subject to input saturation , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[40]  Zhong-Ping Jiang,et al.  Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming , 2016, Autom..

[41]  Gábor Orosz,et al.  Motif-Based Design for Connected Vehicle Systems in Presence of Heterogeneous Connectivity Structures and Time Delays , 2016, IEEE Transactions on Intelligent Transportation Systems.

[42]  Zhong-Ping Jiang,et al.  Robust adaptive dynamic programming for linear and nonlinear systems: An overview , 2013, Eur. J. Control.

[43]  Hannes Hartenstein,et al.  A tutorial survey on vehicular ad hoc networks , 2008, IEEE Communications Magazine.

[44]  D. Kleinman.  On an iterative technique for Riccati equation computations , 1968 .

[45]  João Pedro Hespanha,et al.  Linear Systems Theory , 2009 .

[46]  Joan García-Haro,et al.  Control-based scheduling with QoS support for vehicle to infrastructure communications , 2009, IEEE Wireless Communications.

[47]  Harry L. Trentelman,et al.  Families of linear-quadratic problems: Continuity properties , 1987 .

[48]  K. Chu.  Decentralized Control of High-Speed Vehicular Strings , 1974 .

[49]  Jian-Xin Xu,et al.  Freeway Traffic Control Using Iterative Learning Control-Based Ramp Metering and Speed Signaling , 2007, IEEE Transactions on Vehicular Technology.

[50]  James C. Spall,et al.  TRAFFIC-RESPONSIVE SIGNAL TIMING FOR SYSTEM-WIDE TRAFFIC CONTROL , 1997 .

[51]  Zhong-Ping Jiang,et al.  Sampled-data-based adaptive optimal output-feedback control of a 2-degree-of-freedom helicopter , 2016 .

[52]  Luigi Fortuna,et al.  Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control , 2009 .

[53]  Bart van Arem,et al.  The Impact of Cooperative Adaptive Cruise Control on Traffic-Flow Characteristics , 2006, IEEE Transactions on Intelligent Transportation Systems.

[54]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .