On subspace decompositions of finite horizon DP problems with switched linear dynamics

We consider finite horizon dynamic programming problems for switched linear dynamics over a finite dimensional but otherwise arbitrary state-space, where the cost function depends only on the state. We start from a decomposition of the underlying state-space as the sum of invariant subspaces under all linear transformations induced by the input, and pose the following question: Assuming the cost function has an additive structure compatible with the state-space decomposition, under what conditions does the cost-to-go function also exhibit an additive structure? We begin by deriving a necessary and sufficient condition for the existence of this additive structure. We then propose a sufficient condition, expressed in terms of the cardinality of the input set: While not necessary, this condition has the advantage of being readily verifiable. Finally, we characterize the resulting reduction in complexity for the case of systems with finite state-spaces, and we conclude with a simple illustrative example.

[1]  Ian R. Petersen,et al.  Controllability of quantum systems with switching control , 2011, Int. J. Control.

[2]  Munther A. Dahleh,et al.  A Framework for Robust Stability of Systems Over Finite Alphabets , 2008, IEEE Transactions on Automatic Control.

[3]  Benjamin Van Roy,et al.  An approximate dynamic programming approach to decentralized control of stochastic systems , 2006 .

[4]  V. Belavkin,et al.  Bellman equations for optimal feedback control of qubit states , 2004, quant-ph/0407192.

[5]  Steven Roman Advanced Linear Algebra , 1992 .

[6]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[7]  Munther A. Dahleh,et al.  Finite Approximations of Switched Homogeneous Systems for Controller Synthesis , 2009, IEEE Transactions on Automatic Control.

[8]  O. G. Smolyanov,et al.  Hamilton-Jacobi-Bellman equations for quantum optimal feedback control , 2005, quant-ph/0502155.

[9]  A. Rantzer On Approximate Dynamic Programming in Switching Systems , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[10]  Danielle C. Tarraf,et al.  On subspace decompositions of finite horizon dynamic programming problems , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[11]  R. Laubenbacher,et al.  A computational algebra approach to the reverse engineering of gene regulatory networks. , 2003, Journal of theoretical biology.

[12]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[13]  Kevin T. Phelps,et al.  Coding Theory and Cryptography : The Essentials , 2022 .

[14]  David K. Smith,et al.  Dynamic Programming and Optimal Control. Volume 1 , 1996 .

[15]  Danielle C. Tarraf,et al.  On decompositions of finite horizon DP Problems with linear dynamics , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[16]  Shreyas Sundaram,et al.  Structural Controllability and Observability of Linear Systems Over Finite Fields With Applications to Multi-Agent Systems , 2013, IEEE Transactions on Automatic Control.

[17]  Munther A. Dahleh,et al.  Finite State Controllers for Stabilizing Switched Systems with Binary Sensors , 2007, HSCC.

[18]  Michail G. Lagoudakis,et al.  Least-Squares Policy Iteration , 2003, J. Mach. Learn. Res..

[19]  S. B. Needleman,et al.  A general method applicable to the search for similarities in the amino acid sequence of two proteins. , 1970, Journal of molecular biology.

[20]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[21]  D.C. Tarraf,et al.  Stability of deterministic finite state machines , 2005, Proceedings of the 2005, American Control Conference, 2005..

[22]  Raymond A. DeCarlo,et al.  Optimal control of switching systems , 2005, Autom..

[23]  Danielle C. Tarraf,et al.  A Control-Oriented Notion of Finite State Approximation , 2011, IEEE Transactions on Automatic Control.

[24]  R. Tennant Algebra , 1941, Nature.