Sparsity-Inducing Optimal Control via Differential Dynamic Programming

Optimal control is a popular approach to synthesize highly dynamic motion. Commonly, L2 regularization is used on the control inputs in order to minimize energy used and to ensure smoothness of the control inputs. However, for some systems, such as satellites, the control needs to be applied in sparse bursts due to how the propulsion system operates. In this paper, we study approaches to induce sparsity in optimal control solutions---namely via smooth L1 and Huber regularization penalties. We apply these loss terms to state-of-the-art Differential Dynamic Programming (DDP)-based solvers to create a family of sparsity-inducing optimal control methods. We analyze and compare the effect of the different losses on inducing sparsity, their numerical conditioning, their impact on convergence, and discuss hyperparameter settings. We demonstrate our method in simulation and hardware experiments on canonical dynamics systems, control of satellites, and the NASA Valkyrie humanoid robot. We provide an implementation of our method and all examples for reproducibility on GitHub.

[1]  Edward N. Hartley,et al.  Terminal spacecraft rendezvous and capture with LASSO model predictive control , 2013, Int. J. Control.

[2]  Alexander C. Berg,et al.  RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free , 2019, ArXiv.

[3]  Nicolas Mansard,et al.  Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Nicolas Mansard,et al.  A Direct-Indirect Hybridization Approach to Control-Limited DDP , 2020, ArXiv.

[5]  Mark W. Schmidt,et al.  Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches , 2007, ECML.

[6]  Vladimir Ivan,et al.  Optimizing Dynamic Trajectories for Robustness to Disturbances Using Polytopic Projections , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  David Q. Mayne,et al.  Differential Dynamic Programming–A Unified Approach to the Optimization of Dynamic Systems , 1973 .

[8]  Yuval Tassa,et al.  Control-limited differential dynamic programming , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Danylo Malyuta,et al.  Fast Trajectory Optimization via Successive Convexification for Spacecraft Rendezvous with Integer Constraints , 2019, AIAA Scitech 2020 Forum.

[10]  D. Mayne.  A Second-order Gradient Method for Determining Optimal Trajectories of Non-linear Discrete-time Systems , 1966 .

[11]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  Matteo Parigi Polverini,et al.  A Study on Sparse Hierarchical Inverse Kinematics Algorithms for Humanoid Robots , 2020, IEEE Robotics and Automation Letters.

[14]  Jonas Buchli,et al.  Trajectory Optimization Through Contacts and Automatic Gait Discovery for Quadrupeds , 2016, IEEE Robotics and Automation Letters.

[15]  Yiming Yang,et al.  EXOTica: An Extensible Optimization Toolset for Prototyping and Benchmarking Motion Planning and Control , 2018, Studies in Computational Intelligence.

[16]  Moritz Diehl,et al.  A Family of Iterative Gauss-Newton Shooting Methods for Nonlinear Optimal Control , 2017, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .