Continuous-time finite-horizon ADP for automated vehicle controller design with high efficiency

Automated vehicle controller’s design can be formulated into a general optimal control problem. Existing control methods can not meet the millisecond-level time requirements of onboard standard controllers, especially for nonlinear dynamics with non-affine and saturated controller. This paper presents a continuous-time (CT) finite-horizon approximate dynamic programming (ADP) method, which can synthesis off-line near optimal control policy with analytical vehicle dynamics. Firstly, we develop a finite-horizon variant of the CT Hamilton-Jacobi-Bellman (HJB) equation. To find the nearly optimal solution of the finite-horizon HJB, i.e., the nearly optimal policy, both value function and policy are approximated by neural networks to map the system states to cost-to-go and control inputs respectively. Then the CT finite-horizon ADP algorithm is proposed based on the generalized policy iteration framework to gradually approach the optimal solution of the finite-horizon HJB. The proposed algorithm can converge to the nearly optimal policy and value function. Finally, we apply the proposed algorithm to the automated vehicle trajectory tracking control in both linear and nonlinear cases. The simulation results demonstrate that the proposed finite-horizon ADP algorithm can obtain the nearly optimal control policy and high online operation efficiency (500 times faster than the nonlinear ipopt solver).

[1]  Mario Zanon,et al.  Towards time-optimal race car driving using nonlinear MPC in real-time , 2014, 53rd IEEE Conference on Decision and Control.

[2]  Yanan Li,et al.  Driver-automation indirect shared control of highly automated vehicles with intention-aware authority transition , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[3]  Francesco Borrelli,et al.  MPC-Based Approach to Active Steering for Autonomous Vehicle Systems , 2005 .

[4]  Francesco Borrelli,et al.  Robust nonlinear predictive control for semiautonomous ground vehicles , 2014, 2014 American Control Conference.

[5]  Stephen P. Boyd,et al.  Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding , 2013, Journal of Optimization Theory and Applications.

[6]  Calin Belta,et al.  Experimental Validation of Linear and Nonlinear MPC on an Articulated Unmanned Ground Vehicle , 2018, IEEE/ASME Transactions on Mechatronics.

[7]  J. Christian Gerdes,et al.  Shared Steering Control Using Safe Envelopes for Obstacle Avoidance and Vehicle Stability , 2016, IEEE Transactions on Intelligent Transportation Systems.

[8]  Moritz Diehl,et al.  CasADi: a software framework for nonlinear optimization and optimal control , 2018, Mathematical Programming Computation.

[9]  Shengbo Eben Li,et al.  Generalized Policy Iteration for Optimal Control in Continuous Time , 2019, ArXiv.

[10]  Stephen P. Boyd,et al.  ECOS: An SOCP solver for embedded systems , 2013, 2013 European Control Conference (ECC).

[11]  L. Burns Sustainable mobility: A vision of our transport future , 2013, Nature.

[12]  Paul J. Werbos,et al.  Building and Understanding Adaptive Systems: A Statistical/Numerical Approach to Factory Automation and Brain Research , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Francesco Borrelli,et al.  Kinematic and dynamic vehicle models for autonomous driving control design , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[14]  Stephen P. Boyd,et al.  CVXPY: A Python-Embedded Modeling Language for Convex Optimization , 2016, J. Mach. Learn. Res..

[15]  Zhengyu Liu,et al.  Deep adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraints , 2019, ArXiv.

[16]  Mario Zanon,et al.  Real-Time Constrained Trajectory Planning and Vehicle Control for Proactive Autonomous Driving With Road Users , 2019, 2019 18th European Control Conference (ECC).

[17]  Efstathios Velenis,et al.  A Real-Time Nonlinear Model Predictive Control Strategy for Stabilization of an Electric Vehicle at the Limits of Handling , 2018, IEEE Transactions on Control Systems Technology.

[18]  Stephen P. Boyd,et al.  OSQP: an operator splitting solver for quadratic programs , 2017, 2018 UKACC 12th International Conference on Control (CONTROL).