Optimal control of multiscale systems using reduced-order models

We study optimal control of diffusions with slow and fast variables and address a question raised by practitioners: is it possible to first eliminate the fast variables before solving the optimal control problem and then use the optimal control computed from the reduced-order model to control the original, high-dimensional system? The strategy ``first reduce, then optimize''---rather than ``first optimize, then reduce''---is motivated by the fact that solving optimal control problems for high-dimensional multiscale systems is numerically challenging and often computationally prohibitive. We state sufficient and necessary conditions, under which the ``first reduce, then control'' strategy can be employed and discuss when it should be avoided. We further give numerical examples that illustrate the ``first reduce, then optmize'' approach and discuss possible pitfalls.

[1]  A. R. Norman,et al.  Portfolio Selection with Transaction Costs , 1990, Math. Oper. Res..

[2]  P. Kokotovic.  Applications of Singular Perturbation Techniques to Control Problems , 1984 .

[3]  Alain Bensoussan,et al.  Singular perturbations in stochastic control , 1984 .

[4]  M. Robin.  Long-term average cost control problems for continuous time Markov processes: A survey , 1983 .

[5]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[6]  G. A. Pavliotis,et al.  Maximum likelihood drift estimation for multiscale diffusions , 2008, 0806.3248.

[7]  N. Namachchivaya,et al.  Dimensional reduction in nonlinear filtering , 2010 .

[8]  Petar V. Kokotovic,et al.  Singular perturbation techniques in control theory , 1987 .

[9]  F. Watbled,et al.  On singular perturbations for differential inclusions on the infinite interval , 2005 .

[10]  C. Schütte,et al.  Efficient rare event simulation by optimal nonequilibrium forcing , 2012, 1208.3232.

[11]  Athanasios C. Antoulas,et al.  Approximation of Large-Scale Dynamical Systems , 2005, Advances in Design and Control.

[12]  Ying Hu,et al.  Probabilistic approach to homogenization of viscosity solutions of parabolic PDEs , 1999 .

[13]  C. Holland.  A minimum principle for the principal eigenvalue for second-order linear elliptic equations with natural boundary conditions , 1978 .

[14]  Andreas Steinbrecher,et al.  Optimal Control of Robot Guided Laser Material Treatment , 2010 .

[15]  Carsten Hartmann,et al.  Balanced Truncation of Linear Second-Order Systems: A Hamiltonian Approach , 2010, Multiscale Model. Simul..

[16]  Martino Bardi,et al.  Multiscale problems and homogenization for second-order Hamilton-Jacobi equations , 2007 .

[17]  Carsten Hartmann,et al.  A structure-preserving numerical discretization of reversible diffusions , 2011 .

[18]  A. Antoulas,et al.  A Survey of Model Reduction by Balanced Truncation and Some New Results , 2004 .

[19]  G. Nguetseng.  A general convergence result for a functional related to the theory of homogenization , 1989 .

[20]  A. Bensoussan.  Perturbation Methods in Optimal Control , 1988 .

[21]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[22]  A. Bensoussan,et al.  An ergodic control problem arising from the principal eigenfunction of an elliptic operator , 1991 .

[23]  K. Glover.  All optimal Hankel-norm approximations of linear multivariable systems and their L, ∞ -error bounds† , 1984 .

[24]  P. Kokotovic,et al.  Singular perturbations and asymptotic analysis in control systems , 1986 .

[25]  H. Kushner.  Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems , 1990 .

[26]  Carsten Hartmann,et al.  Balanced Averaging of Bilinear Systems with Applications to Stochastic Control , 2013, SIAM J. Control. Optim..

[27]  Naoyuki Ichihara,et al.  A Stochastic Representation for Fully Nonlinear PDEs and Its Application to Homogenization , 2005 .

[28]  B. Moore.  Principal component analysis in linear systems: Controllability, observability, and model reduction , 1981 .

[29]  B. Øksendal.  Stochastic differential equations : an introduction with applications , 1987 .

[30]  L. Evans.  The perturbed test function method for viscosity solutions of nonlinear PDE , 1989, Proceedings of the Royal Society of Edinburgh: Section A Mathematics.

[31]  S. Pergamenshchikov,et al.  Two-Scale Stochastic Systems: Asymptotic Analysis and Control , 2010 .

[32]  Martino Bardi,et al.  Viscosity Solutions Methods for Singular Perturbations in Deterministic and Stochastic Control , 2001, SIAM J. Control. Optim..

[33]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[34]  Zvi Artstein,et al.  ON SINGULARLY PERTURBED ORDINARY DIFFERENTIAL EQUATIONS WITH MEASURE-VALUED LIMITS , 2002 .

[35]  Harold J. Kushner.  Direct averaging and perturbed test function methods for weak convergence , 1986 .

[36]  Konstantinos Spiliopoulos,et al.  Importance Sampling for Multiscale Diffusions , 2011, Multiscale Model. Simul..

[37]  Panagiotis E. Souganidis,et al.  Correctors for the homogenization of Hamilton‐Jacobi equations in the stationary ergodic setting , 2003 .

[38]  V. Gaitsgory.  Suboptimization of singularly perturbed control systems , 1992 .

[39]  Martino Bardi,et al.  MULTISCALE SINGULAR PERTURBATIONS AND HOMOGENIZATION OF OPTIMAL CONTROL PROBLEMS , 2007 .

[40]  G. A. Pavliotis,et al.  Parameter Estimation for Multiscale Diffusions , 2007 .

[41]  J. Mount.  Importance Sampling , 2005 .

[42]  J. Bismut.  Martingales, the Malliavin calculus and hypoellipticity under general Hörmander's conditions , 1981 .

[43]  A. Vigodner.  Limits of Singularly Perturbed Control Problems with Statistical Dynamics of Fast Motions , 1997 .

[44]  Zoran Gajic,et al.  Optimal Control Of Singularly Perturbed Linear Systems And Applications , 2001 .

[45]  P. Dupuis,et al.  Importance Sampling, Large Deviations, and Differential Games , 2004 .

[46]  G. Grammel.  Averaging of singularly perturbed systems , 1997 .

[47]  A. Bensoussan,et al.  Asymptotic analysis for periodic structures , 1979 .

[48]  Wolfgang J. Runggaldier,et al.  Connections between stochastic control and dynamic games , 1996, Math. Control. Signals Syst..

[49]  Thorsten Klüner,et al.  Optimal control of open quantum systems applied to the photochemistry of surfaces. , 2011, Physical review letters.

[50]  Carsten Hartmann,et al.  Balanced model reduction of partially observed Langevin equations: an averaging principle , 2011 .

[51]  Ying Hu,et al.  Probabilistic approach to homogenizations of systems of quasilinear parabolic PDEs with periodic structures , 1998 .

[52]  P. Imkeller,et al.  Dimensional reduction in nonlinear filtering: A homogenization approach , 2011, 1112.2986.

[53]  Edward Nelson.  Dynamical Theories of Brownian Motion , 1967 .

[54]  Huyen Pham,et al.  Continuous-time stochastic control and optimization with financial applications / Huyen Pham , 2009 .

[55]  Henrik Stapelfeldt,et al.  Laser Aligned Molecules: Applications in Physics and Chemistry , 2004 .

[56]  T. Kurtz,et al.  Stationary Solutions and Forward Equations for Controlled and Singular Martingale Problems , 2001 .

[57]  Grigorios A. Pavliotis,et al.  Multiscale Methods: Averaging and Homogenization , 2008 .

[58]  Carsten Hartmann,et al.  Optimal control of molecular dynamics using Markov state models , 2012, Math. Program..

[59]  J. Zabczyk,et al.  Exit problem and control theory , 1985 .

[60]  P. Malliavin.  Stochastic calculus of variation and hypoelliptic operators , 1978 .

[61]  Paul Van Dooren,et al.  A collection of benchmark examples for model reduction of linear time invariant dynamical systems. , 2002 .