Reinforcement learning of dynamic collaborative driving Part II: lateral adaptive control

Dynamic collaborative driving involves the motion coordination of multiple vehicles using shared information from vehicles instrumented to perceive their surroundings in order to improve road usage and safety. A basic requirement of any vehicle participating in dynamic collaborative driving is longitudinal control. Without this capability, higher-level coordination is not possible. Each vehicle involved is a composite non-linear system powered by an internal combustion engine, equipped with automatic transmission, rolling on rubber tyres with a hydraulic braking system. This paper focuses on the problem of longitudinal motion control. A longitudinal vehicle model is introduced which serves as the control system design platform. A longitudinal adaptive control system that uses Monte Carlo Reinforcement Learning (RL) is introduced. The results of the RL phase and the performance of the adaptive control system for a single automobile, as well as the performance in a multi-vehicle platoon, are presented.

[1]  Masayoshi Tomizuka,et al.  Integrated Maneuvering Control For Automated Highway Systems Based On A Magnetic Reference/sensing System , 1995 .

[2]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[3]  Jürgen Guldner,et al.  Study of design directions for lateral vehicle control , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[4]  Jürgen Guldner,et al.  Robust automatic steering control for look-down reference systems with front and rear sensors , 1999, IEEE Trans. Control. Syst. Technol..

[5]  Petros A. Ioannou,et al.  Autonomous intelligent cruise control , 1993 .

[6]  Will Recker,et al.  An analytical model of multihop connectivity of inter-vehicle communication systems , 2010, IEEE Transactions on Wireless Communications.

[7]  Wen-Long Jin,et al.  Instantaneous Information Propagation in a Traffic Stream Through Inter-Vehicle Communication , 2006 .

[8]  Wei-Bin Zhang,et al.  Demonstration of integrated longitudinal and lateral control for the operation of automated vehicles in platoons , 2000, IEEE Trans. Control. Syst. Technol..

[9]  François Michaud,et al.  Coordinated Maneuvering of Automated Vehicles in Platoons , 2006, IEEE Transactions on Intelligent Transportation Systems.

[10]  Karl Johan Åström,et al.  Adaptive Control , 1989, Embedded Digital Control with Microcontrollers.

[11]  Robert E. Fenton,et al.  On the steering of automated vehicles: Theory and experiment , 1976 .

[12]  Xu Yang,et al.  Simulation studies of information propagation in a self-organizing distributed traffic information system , 2005 .

[13]  Swaroop Darbha,et al.  Direct adaptive longitudinal control of vehicle platoons , 2001, IEEE Trans. Veh. Technol..

[14]  M.E. Khatir,et al.  A decentralized lateral-longitudinal controller for a platoon of vehicles operating on a plane , 2006, 2006 American Control Conference.

[15]  Gary J. Heydinger,et al.  Model Validation of the 1997 Jeep Cherokee for the National Advanced Driving Simulator , 2000 .

[16]  Jiang Zhang,et al.  A Zero Public Infrastructure Vehicle Based Traffic Information System , 2003 .

[17]  Keith Gardels.  AUTOMATIC CAR CONTROLS FOR ELECTRONIC HIGHWAYS , 1960 .

[18]  Wei-bin Zhang,et al.  An Intelligent Roadway Reference System for Vehicle Lateral Guidance/Control , 1990, 1990 American Control Conference.

[19]  R. Bellman.  A Markovian Decision Process , 1957 .

[20]  Azim Eskandarian,et al.  Challenges of intervehicle ad hoc networks , 2004, IEEE Transactions on Intelligent Transportation Systems.

[21]  M.S. Netto,et al.  Lateral adaptive control for vehicle lane keeping , 2004, Proceedings of the 2004 American Control Conference.

[22]  Petros A. Ioannou,et al.  Adaptive Vehicle Following Control System with Variable Time Headways , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[23]  Luke Ng,et al.  Reinforcement Learning of Dynamic Collaborative Driving , 2008 .

[24]  Ümit Özgüner,et al.  Design and stability analysis of a lane following controller , 1996, IEEE Trans. Control. Syst. Technol..

[25]  Xu Yang,et al.  Modeling Dynamic Vehicle Navigation in a Self-Organizing, Peer-to-Peer, Distributed Traffic Information System , 2006, J. Intell. Transp. Syst..

[26]  Dean A. Pomerleau,et al.  The 1997 automated highway free agent demonstration , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[27]  R.E. Fenton,et al.  On the optimal design of a vehicle lateral controller , 1984, 34th IEEE Vehicular Technology Conference.

[28]  Hans B. Pacejka,et al.  Tyre Modelling for Use in Vehicle Dynamics Studies , 1987 .

[29]  Hannes Hartenstein,et al.  Position-aware ad hoc wireless networks for inter-vehicle communications: the Fleetnet project , 2001, MobiHoc '01.

[30]  Nick McKeown,et al.  Automated vehicle control developments in the PATH program , 1991 .

[31]  S Kato,et al.  An architecture for cooperative driving of automated vehicles , 2000, ITSC2000. 2000 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.00TH8493).

[32]  Sadayuki Tsugawa,et al.  Cooperative driving of automated vehicles with inter-vehicle communications , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[33]  Masayoshi Tomizuka,et al.  Combined lateral and longitudinal control of vehicles for IVHS , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[34]  M. Rudack.  On the Dynamics of Ad Hoc Networks for Inter Vehicle Communications ( IVC ) , 2002 .

[35]  L. Briesemeister,et al.  Disseminating messages among highly mobile hosts based on inter-vehicle communication , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[36]  J. K. Hedrick,et al.  A robust optimal lateral vehicle control strategy , 1996, Proceeding of the 1996 IEEE International Conference on Control Applications IEEE International Conference on Control Applications held together with IEEE International Symposium on Intelligent Contro.

[37]  Tsu Chin,et al.  California Partners for Advanced Transit and Highways (PATH) , 2004 .

[38]  Dean A. Pomerleau,et al.  PANS: a portable navigation platform , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[39]  Masayoshi Tomizuka,et al.  Experimental Automatic Lateral Control System for an Automobile , 1992 .

[40]  E J Haug,et al.  FEASABILITY STUDY AND CONCEPTUAL DESIGN OF A NATIONAL ADVANCED DRIVING SIMULATOR , 1990 .

[41]  M. Tomizuka,et al.  Control issues in automated highway systems , 1994, IEEE Control Systems.

[42]  Petros A. Ioannou,et al.  Vehicle following control design for automated highway systems , 1997, 1997 IEEE 47th Vehicular Technology Conference. Technology in Motion.

[43]  J. K. Hedrick,et al.  Longitudinal Model Development For Automated Roadway Vehicles , 1990 .

[44]  Vadim I. Utkin,et al.  A sliding mode control approach to automatic car steering , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[45]  Rajesh Rajamani,et al.  Demonstration of an automated highway platoon system , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[46]  R. E. Fenton,et al.  On the optimal design of an automotive lateral controller , 1988 .

[47]  Masayoshi Tomizuka,et al.  Optimal Preview Control For Vehicle Lateral Guidance , 1991 .

[48]  Paul A. Grygier,et al.  Model Validation of the 1998 Chevrolet Malibu for the National Advanced Driving Simulator , 2001 .

[49]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[50]  Pravin Varaiya,et al.  Smart cars on smart roads: problems of control , 1991, IEEE Trans. Autom. Control..