Constraint Handling in Continuous-Time DDP-Based Model Predictive Control

The Sequential Linear Quadratic (SLQ) algorithm is a continuous-time variant of the well-known Differential Dynamic Programming (DDP) technique with a Gauss-Newton Hessian approximation. This family of methods has gained popularity in the robotics community due to its efficiency in solving complex trajectory optimization problems. However, one major drawback of DDP-based formulations is their inability to properly incorporate path constraints. In this paper, we address this issue by devising a constrained SLQ algorithm that handles a mixture of constraints with a previously implemented projection technique and a new augmented-Lagrangian approach. By providing an appropriate multiplier update law, and by solving a single inner and outer loop iteration, we are able to retrieve suboptimal solutions at rates suitable for real-time model-predictive control applications. We particularly focus on the inequality-constrained case, where three augmentedLagrangian penalty functions are introduced, along with their corresponding multiplier update rules. These are then benchmarked against a relaxed log-barrier formulation in a cart-pole swing up example, an obstacle-avoidance task, and an objectpushing task with a quadrupedal mobile manipulator.

[1]  Ryan P. Russell,et al.  A Hybrid Differential Dynamic Programming Algorithm for Constrained Optimal Control Problems. Part 1: Theory , 2012, Journal of Optimization Theory and Applications.

[2]  Edo Jelavic,et al.  Real-time motion planning of legged robots: A model predictive control approach , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[3]  Yuval Tassa,et al.  Control-limited differential dynamic programming , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Marco Hutter,et al.  A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation , 2021, IEEE Robotics and Automation Letters.

[5]  William D. Smart,et al.  Receding Horizon Differential Dynamic Programming , 2007, NIPS.

[6]  Stephen P. Boyd,et al.  Fast Model Predictive Control Using Online Optimization , 2010, IEEE Transactions on Control Systems Technology.

[7]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[8]  Christian Feller,et al.  Relaxed Logarithmic Barrier Function Based Model Predictive Control of Linear Systems , 2015, IEEE Transactions on Automatic Control.

[9]  Jonas Buchli,et al.  An efficient optimal planning and control framework for quadrupedal locomotion , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Scott Kuindersma,et al.  Constrained unscented dynamic programming , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Alexander Domahidi,et al.  FORCES NLP: an efficient implementation of interior-point methods for multistage nonlinear nonconvex programs , 2020, Int. J. Control.

[12]  John T. Betts,et al.  Practical Methods for Optimal Control and Estimation Using Nonlinear Programming , 2009 .

[13]  James E. Bobrow,et al.  An efficient sequential linear quadratic algorithm for solving nonlinear optimal control problems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[14]  Emanuel Todorov,et al.  Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems , 2004, ICINCO.

[15]  Sangbae Kim,et al.  Dynamic Locomotion in the MIT Cheetah 3 Through Convex Model-Predictive Control , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Ruben Grandia,et al.  Feedback MPC for Torque-Controlled Legged Robots , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Yuval Tassa,et al.  Synthesis and stabilization of complex behaviors through online trajectory optimization , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Olivier Stasse,et al.  Whole-body model-predictive control applied to the HRP-2 humanoid , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[20]  C. Karen Liu,et al.  Differential dynamic programming with nonlinear constraints , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Zachary Manchester,et al.  ALTRO: A Fast Solver for Constrained Trajectory Optimization , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Marco Hutter,et al.  Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupeds , 2017, IEEE Robotics and Automation Letters.

[23]  MORITZ DIEHL,et al.  A Real-Time Iteration Scheme for Nonlinear Optimization in Optimal Feedback Control , 2005, SIAM J. Control. Optim..

[24]  M. Hestenes Multiplier and gradient methods , 1969 .

[25]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.

[26]  José Mario Martínez,et al.  Numerical Comparison of Augmented Lagrangian Algorithms for Nonconvex Problems , 2005, Comput. Optim. Appl..

[27]  Roland Siegwart,et al.  Fast nonlinear Model Predictive Control for unified trajectory optimization and tracking , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Marc Toussaint A Novel Augmented Lagrangian Approach for Inequalities and Convergent Any-Time Non-Central Updates , 2014, 1412.4329.

[29]  Evangelos A. Theodorou,et al.  Constrained Differential Dynamic Programming Revisited , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .