Superlinear Convergence Using Controls Based on Second-Order Needle Variations

This paper investigates the convergence performance of second-order needle variation methods for nonlinear control-affine systems. Control solutions have a closed-form expression that is derived from the first-and second-order mode insertion gradients of the objective and are proven to exhibit superlinear convergence near equilibrium. Compared to first-order needle variations, the proposed synthesis scheme exhibits superior convergence at smaller computational cost than alternative nonlinear feedback controllers. Simulation results on the differential drive model verify the analysis and show that second-order needle variations outperform first-order variational methods and iLQR near the optimizer. Last, even when implemented in a closed-loop, receding horizon setting, the proposed algorithm demonstrates superior convergence against the iterative linear quadratic Gaussian (iLQG) controller.

[1]  Kathrin Flaßkamp,et al.  Sequential Action Control for Tracking of Free Invariant Manifolds , 2015, ADHS.

[2]  Todd Murphey,et al.  Online Feedback Control for Input-Saturated Robotic Systems on Lie Groups , 2016, Robotics: Science and Systems.

[3]  Frank L. Lewis,et al.  Hybrid control for a class of underactuated mechanical systems , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[4]  Y. Wardi,et al.  Algorithm for optimal mode scheduling in switched systems , 2012, 2012 American Control Conference (ACC).

[5]  Alberto Bemporad,et al.  Model predictive control based on linear programming - the explicit solution , 2002, IEEE Transactions on Automatic Control.

[6]  Wei Sun,et al.  Game Theoretic continuous time Differential Dynamic Programming , 2015, 2015 American Control Conference (ACC).

[7]  F. Allgöwer,et al.  Nonlinear Model Predictive Control: From Theory to Application , 2004 .

[8]  Todd D. Murphey,et al.  Projection-based optimal mode scheduling , 2013, 52nd IEEE Conference on Decision and Control.

[9]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[10]  Emanuel Todorov,et al.  Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems , 2004, ICINCO.

[11]  Todd D. Murphey,et al.  Model-Based Reactive Control for Hybrid and High-Dimensional Robotic Systems , 2016, IEEE Robotics and Automation Letters.

[12]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[13]  S. Yakowitz,et al.  Differential dynamic programming and Newton's method for discrete optimal control problems , 1984 .

[14]  Wei Sun,et al.  Continuous-time differential dynamic programming with terminal constraints , 2014, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[15]  Magnus Egerstedt,et al.  Transition-time optimization for switched-mode dynamical systems , 2006, IEEE Transactions on Automatic Control.

[16]  B. Anderson,et al.  Optimal control: linear quadratic methods , 1990 .

[17]  Y. Wardi,et al.  OPTIMAL CONTROL OF SWITCHING TIMES IN HYBRID SYSTEMS , 2003 .

[18]  Mohd Ashraf Ahmad,et al.  PERFORMANCE COMPARISON BETWEEN LQR AND PID CONTROLLERS FOR AN INVERTED PENDULUM SYSTEM , 2008 .

[19]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[20]  Magnus Egerstedt,et al.  A controlled-precision algorithm for mode-switching optimization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[21]  J. Hauser.  A PROJECTION OPERATOR APPROACH TO THE OPTIMIZATION OF TRAJECTORY FUNCTIONALS , 2002 .

[22]  Todd D. Murphey,et al.  Real-time trajectory synthesis for information maximization using Sequential Action Control and least-squares estimation , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Todd D. Murphey,et al.  Feedback synthesis for underactuated systems using sequential second-order needle variations , 2018, Int. J. Robotics Res..

[24]  Todd D. Murphey,et al.  Sequential Action Control: Closed-Form Optimal Control for Nonlinear and Nonsmooth Systems , 2016, IEEE Transactions on Robotics.

[25]  P.V. Kokotovic,et al.  The joy of feedback: nonlinear and adaptive , 1992, IEEE Control Systems.

[26]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[27]  Todd D. Murphey,et al.  Sequential Action Control for models of underactuated underwater vehicles in a planar ideal fluid , 2016, 2016 American Control Conference (ACC).

[28]  Katsuhisa Furuta,et al.  Swinging up a pendulum by energy control , 1996, Autom..

[29]  Phillipp Kaestner,et al.  Linear And Nonlinear Programming , 2016 .

[30]  M. Leok,et al.  Controlled Lagrangians and Stabilization of the Discrete Cart-Pendulum System , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[31]  Todd D. Murphey,et al.  Switching mode generation and optimal estimation with application to skid-steering , 2011, Autom..