Aggressive driving with model predictive path integral control

In this paper we present a model predictive control algorithm designed for optimizing non-linear systems subject to complex cost criteria. The algorithm is based on a stochastic optimal control framework using a fundamental relationship between the information theoretic notions of free energy and relative entropy. The optimal controls in this setting take the form of a path integral, which we approximate using an efficient importance sampling scheme. We experimentally verify the algorithm by implementing it on a Graphics Processing Unit (GPU) and apply it to the problem of controlling a fifth-scale Auto-Rally vehicle in an aggressive driving task.

[1]  H. Kappen.  Linear theory for control of nonlinear stochastic systems. , 2004, Physical review letters.

[2]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[3]  Efstathios Velenis,et al.  Modeling aggressive maneuvers on loose surfaces: The cases of Trail-Braking and Pendulum-Turn , 2007, 2007 European Control Conference (ECC).

[4]  William Whittaker,et al.  Tartan Racing: A multi-modal approach to the DARPA Urban Challenge , 2007 .

[5]  P. Tsiotras,et al.  Minimum-Time Travel for a Vehicle with Acceleration Limits: Theoretical Analysis and Receding-Horizon Implementation , 2008 .

[6]  M. Gerdts,et al.  Generating locally optimal trajectories for an automatically driven car , 2009 .

[7]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[8]  Kirstin L. R. Talvala,et al.  Pushing the limits: From lanekeeping to autonomous racing , 2011, Annu. Rev. Control..

[9]  Frank Dellaert,et al.  iSAM2: Incremental smoothing and mapping using the Bayes tree , 2012, Int. J. Robotics Res..

[10]  Marc Toussaint,et al.  On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference , 2012, Robotics: Science and Systems.

[11]  Evangelos Theodorou,et al.  Relative entropy and free energy dualities: Connections to Path Integral and KL control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[12]  J. Christian Gerdes,et al.  Using the centre of percussion to design a steering controller for an autonomous race car , 2012 .

[13]  Gabe Sibley,et al.  Realtime Simulation-in-the-Loop Control for Agile Ground Vehicles , 2013, TAROS.

[14]  Y. Matsuoka,et al.  Reinforcement Learning and Synergistic Control of the ACT Hand , 2013, IEEE/ASME Transactions on Mechatronics.

[15]  Vicenç Gómez,et al.  Policy Search for Path Integral Control , 2014, ECML/PKDD.

[16]  Panagiotis Tsiotras,et al.  Real-Time Near-Optimal Feedback Control of Aggressive Vehicle Maneuvers , 2014 .

[17]  Han Wang,et al.  Applications of the Cross-Entropy Method to Importance Sampling and Optimal Control of Diffusions , 2014, SIAM J. Sci. Comput..

[18]  Eric Rombokas,et al.  GPU Based Path Integral Control with Learned Dynamics , 2015, ArXiv.

[19]  Frank Dellaert,et al.  IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation , 2015, Robotics: Science and Systems.

[20]  Evangelos Theodorou,et al.  Model Predictive Path Integral Control using Covariance Variable Importance Sampling , 2015, ArXiv.

[21]  Evangelos Theodorou,et al.  Nonlinear Stochastic Control and Information Theoretic Dualities: Connections, Interdependencies and Thermodynamic Interpretations , 2015, Entropy.

[22]  Vicenç Gómez,et al.  Real-Time Stochastic Optimal Control for Multi-Agent Quadrotor Systems , 2015, ICAPS.