Symmetry reduction for dynamic programming and application to MRI

We present a method of exploiting symmetries of discrete-time optimal control problems to reduce the dimensionality of dynamic programming iterations. The results are derived for systems with continuous state variables, and can be applied to systems with continuous or discrete symmetry groups. We prove that symmetries of the state update equation and stage costs induce corresponding symmetries of the optimal cost function and the optimal policies. Thus symmetries can be exploited to allow dynamic programming iterations to be performed in a reduced state space. The application of these results is illustrated using a model of spin dynamics for magnetic resonance imaging (MRI). For this application problem, the symmetry reduction introduced leads to a significant speedup, reducing computation time by a factor of 75×.

[1]  A. Prasad Sistla,et al.  Symmetry and model checking , 1993, Formal Methods Syst. Des..

[2]  Tucker R. Balch,et al.  Symmetry in Markov Decision Processes and its Implications for Single Agent and Multiagent Learning , 2001, ICML.

[3]  Peder E. Z. Larson,et al.  Optimal experiment design for physiological parameter estimation using hyperpolarized carbon-13 magnetic resonance imaging , 2015, 2015 American Control Conference (ACC).

[4]  Elizabeth L. Mansfield,et al.  Symmetry reductions and exact solutions of a class of nonlinear heat equations , 1993, solv-int/9306002.

[5]  Axel Barrau,et al.  Intrinsic Filtering on Lie Groups With Applications to Attitude Estimation , 2013, IEEE Transactions on Automatic Control.

[6]  S. Marcus,et al.  Optimal control of systems possessing symmetries , 1984 .

[7]  Balaraman Ravindran,et al.  Efficiently Exploiting Symmetries in Real Time Dynamic Programming , 2007, IJCAI.

[8]  Richard M. Murray,et al.  Tracking for fully actuated mechanical systems: a geometric framework , 1999, Autom..

[9]  W. W. Hansen,et al.  The Nuclear Induction Experiment , 1946 .

[10]  Pierre Vandergheynst,et al.  A Compressed Sensing Framework for Magnetic Resonance Fingerprinting , 2013, SIAM J. Imaging Sci..

[11]  Issa Amadou Tall,et al.  Nonlinearizable single-input control systems do not admit stationary symmetries , 2002, Syst. Control. Lett..

[12]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[13]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[14]  Magnus Egerstedt,et al.  Controllability of Multi-Agent Systems from a Graph-Theoretic Perspective , 2009, SIAM J. Control. Optim..

[15]  P. Krishnaprasad,et al.  Nonholonomic mechanical systems with symmetry , 1996 .

[16]  Axel Barrau,et al.  The Invariant Extended Kalman Filter as a Stable Observer , 2014, IEEE Transactions on Automatic Control.

[17]  J. Duerk,et al.  Magnetic Resonance Fingerprinting , 2013, Nature.

[18]  W. W. Hansen,et al.  Nuclear Induction , 2011 .

[19]  A. O. Rodríguez,et al.  Principles of magnetic resonance imaging , 2004 .

[20]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[21]  Murat Arcak,et al.  Symmetry Reduction for Performance Certification of Interconnected Systems , 2018, IEEE Transactions on Control of Network Systems.

[22]  Warren B. Powell,et al.  Perspectives of approximate dynamic programming , 2012, Annals of Operations Research.

[23]  John N. Maidens,et al.  Parallel dynamic programming for optimal experiment design in nonlinear systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[24]  Jeremy W. Gordon,et al.  Optimizing Flip Angles for Metabolic Rate Estimation in Hyperpolarized Carbon-13 MRI , 2016, IEEE Transactions on Medical Imaging.

[25]  Marta Z. Kwiatkowska,et al.  Symmetry Reduction for Probabilistic Model Checking , 2006, CAV.

[26]  R. Bellman,et al.  FUNCTIONAL APPROXIMATIONS AND DYNAMIC PROGRAMMING , 1959 .

[27]  Tomoki Ohsawa.  Symmetry Reduction of Optimal Control Systems and Principal Connections , 2013, SIAM J. Control. Optim..

[28]  John N. Maidens,et al.  Semidefinite relaxations in optimal experiment design with application to substrate injection for hyperpolarized MRI , 2015, 2016 American Control Conference (ACC).