Feedback Solution to Optimal Switching Problems With Switching Cost

The problem of optimal switching between nonlinear autonomous subsystems is investigated in this paper where the objective is not only bringing the states to close to the desired point, but also adjusting the switching pattern, in the sense of penalizing switching occurrences and assigning different preferences to utilization of different modes. The mode sequence is unspecified and a switching cost term is used in the cost function for penalizing each switching. It is shown that once a switching cost is incorporated, the optimal cost-to-go function depends on the subsystem which was active at the previous time step. Afterward, an approximate dynamic programming-based method is developed, which provides an approximation of the optimal solution to the problem in a feedback form and for different initial conditions. Finally, the performance of the method is analyzed through numerical examples.

[1]  Donald E. Kirk,et al.  Optimal control theory : an introduction , 1970 .

[2]  F. Lewis,et al.  Discrete-time nonlinear HJB solution using Approximate dynamic programming: Convergence Proof , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[3]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  S. N. Balakrishnan,et al.  Adaptive-critic based neural networks for aircraft optimal control , 1996 .

[5]  Ali Heydari,et al.  Optimal switching between controlled subsystems with free mode sequence , 2015, Neurocomputing.

[6]  Maryam Kamgarpour,et al.  On optimal control of non-autonomous switched systems with a fixed mode sequence , 2012, Autom..

[7]  Frank L. Lewis,et al.  Reinforcement Learning and Approximate Dynamic Programming for Feedback Control , 2012 .

[8]  Rui-yan Zhao,et al.  Switched system optimal control based on parameterizations of the control vectors and switching instants , 2011, 2011 Chinese Control and Decision Conference (CCDC).

[9]  Huaguang Zhang,et al.  Finite horizon optimal control of non-linear discrete-time switched systems using adaptive dynamic programming with ε-error bound , 2014, Int. J. Syst. Sci..

[10]  Ali Heydari,et al.  Finite-Horizon Control-Constrained Nonlinear Optimal Control Using Single Network Adaptive Critics , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[11]  M.R. James,et al.  On Computation of Optimal Switching HJB Equation , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[12]  Ali Heydari,et al.  Revisiting Approximate Dynamic Programming and its Convergence , 2014, IEEE Transactions on Cybernetics.

[13]  Ali Heydari,et al.  Optimal Switching and Control of Nonlinear Switching Systems Using Approximate Dynamic Programming , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Panos J. Antsaklis,et al.  Optimal control of switched systems based on parameterization of the switching instants , 2004, IEEE Transactions on Automatic Control.

[15]  Mohamed Benrejeb,et al.  Optimization of switching instants for optimal control of linear switched systems based on genetic algorithms , 2009, ICONS.

[16]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[17]  Derong Liu,et al.  Adaptive Dynamic Programming for Control: Algorithms and Stability , 2012 .

[18]  Ali Heydari,et al.  Fixed-final-time optimal control of nonlinear systems with terminal constraints , 2013, Neural Networks.

[19]  Liyan Zhang,et al.  Optimal Control of Switched System Based on Neural Network Optimization , 2008, ICIC.

[20]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[21]  O. Stursberg,et al.  A numerical method for hybrid optimal control based on dynamic programming , 2011 .

[22]  Magnus Egerstedt,et al.  Real-time optimal feedback control of switched autonomous systems , 2009, ADHS.

[23]  Wenjie Lu,et al.  An approximate dynamic programming approach for model-free control of switched systems , 2013, 52nd IEEE Conference on Decision and Control.

[24]  Mark Weiser,et al.  Source Code , 1987, Computer.

[25]  Paolo Valigi,et al.  Optimal mode-switching for hybrid systems with varying initial states , 2008 .

[26]  A. Heydari,et al.  Optimal Orbit Transfer with ON-OFF Actuators Using a Closed Form Optimal Switching Scheme , 2013 .

[27]  Derong Liu,et al.  Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems , 2013, IEEE Transactions on Cybernetics.

[28]  Bevan K. Youse,et al.  Introduction to real analysis , 1972 .

[29]  Sarangapani Jagannathan,et al.  Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence , 2009, Neural Networks.

[30]  Hao Xu,et al.  Optimal control of uncertain quantized linear discrete‐time systems , 2015 .

[31]  Xuping Xu,et al.  Optimal control of switched systems via non-linear optimization based on direct differentiations of value functions , 2002 .

[32]  Magnus Egerstedt,et al.  Transition-time optimization for switched-mode dynamical systems , 2006, IEEE Transactions on Automatic Control.

[33]  W. Dixon.  Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles , 2014 .

[34]  Ali Heydari,et al.  Optimal switching between autonomous subsystems , 2014, J. Frankl. Inst..

[35]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[36]  Derong Liu,et al.  An iterative ϵ-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state , 2012, Neural Networks.

[37]  Ali Heydari,et al.  Optimal multi-therapeutic HIV treatment using a global optimal switching scheme , 2013, Appl. Math. Comput..

[38]  H. Axelsson,et al.  Algorithm for Switching-Time Optimization in Hybrid Dynamical Systems , 2005, Proceedings of the 2005 IEEE International Symposium on, Mediterrean Conference on Control and Automation Intelligent Control, 2005..