Optimal Steering of a Linear Stochastic System to a Final Probability Distribution—Part III

The subject of this work has its roots in the so-called Schrödginer bridge problem (SBP) which asks for the most likely distribution of Brownian particles in their passage between observed empirical marginal distributions at two distinct points in time. Renewed interest in this problem was sparked by a reformulation in the language of stochastic control. In earlier works, presented as Part I and Part II, we explored a generalization of the original SBP that amounts to optimal steering of linear stochastic dynamical systems between state-distributions, at two points in time, under full state feedback. In these works, the cost was quadratic in the control input, i.e., control energy. The purpose of the present work is to detail the technical steps in extending the framework to the case where a quadratic cost in the state is also present. Thus, the main contribution is to derive the optimal control in this case which in fact is given in closed-form (Theorem 1). In the zero-noise limit, we also obtain the solution of a (deterministic) mass transport problem with general quadratic cost.

[1]  J Barhen,et al.  Control of friction at the nanoscale. , 2003, Physical review letters.

[2]  M. Miles,et al.  High-Q dynamic force microscopy in liquid and its application to living cells. , 2001, Biophysical journal.

[3]  M Ono,et al.  Chance constrained finite horizon optimal control with nonconvex constraints , 2010, Proceedings of the 2010 American Control Conference.

[4]  G. Prodi,et al.  Feedback cooling of the normal modes of a massive electromechanical system to submillikelvin temperature. , 2008, Physical review letters.

[5]  Yuan,et al.  Thermal noise reduction of mechanical oscillators by actively controlled external dissipative forces , 2000, Ultramicroscopy.

[6]  A. Blaquière,et al.  Controllability of a Fokker-Planck equation, the Schrödinger system, and a related stochastic optimal control (revised version) , 1992 .

[7]  N C Petroni,et al.  Stochastic collective dynamics of charged-particle beams in the stability regime. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  H. V. D. Zant,et al.  Mechanical systems in the quantum regime , 2011, 1106.2060.

[9]  E. Schrödinger Sur la théorie relativiste de l'électron et l'interprétation de la mécanique quantique , 1932 .

[10]  Tryphon T. Georgiou,et al.  Optimal steering of inertial particles diffusing anisotropically with losses , 2015, 2015 American Control Conference (ACC).

[11]  Edward Nelson Dynamical Theories of Brownian Motion , 1967 .

[12]  Toshio Mikami,et al.  Optimal Transportation Problem by Stochastic Optimal Control , 2005, SIAM J. Control. Optim..

[13]  M. Pavon,et al.  Discrete-time classical and quantum Markovian evolutions: Maximum entropy problems on path space , 2008, 0811.0933.

[14]  Max-Olivier Hongler,et al.  Relative entropy and efficiency measure for diffusion-mediated transport processes , 2005 .

[15]  Max-Olivier Hongler,et al.  Connection between an Exactly Solvable Stochastic Optimal Control Problem and a Nonlinear Reaction-Diffusion Equation , 2008 .

[16]  W. Fleming Logarithmic Transformations and Stochastic Control , 1982 .

[17]  A. Beghi,et al.  On the relative entropy of discrete-time Markov processes with given end-point densities , 1996, IEEE Trans. Inf. Theory.

[18]  Tryphon T. Georgiou,et al.  On the Relation Between Optimal Transport and Schrödinger Bridges: A Stochastic Control Viewpoint , 2014, J. Optim. Theory Appl..

[19]  Matthew Pitkin,et al.  Gravitational Wave Detection by Interferometry (Ground and Space) , 2000, Living reviews in relativity.

[20]  Tryphon T. Georgiou,et al.  Optimal Transport Over a Linear Dynamical System , 2015, IEEE Transactions on Automatic Control.

[21]  A. Beurling,et al.  An Automorphism of Product Measures , 1960 .

[22]  Zhi-Quan Luo,et al.  Geometric Methods for Spectral Analysis , 2012, IEEE Transactions on Signal Processing.

[23]  Shoichi Toyabe,et al.  Nonequilibrium energetics of a single F1-ATPase molecule. , 2010, Physical review letters.

[24]  P. Reimann Brownian motors: noisy transport far from equilibrium , 2000, cond-mat/0010237.

[25]  Tryphon T. Georgiou,et al.  Stochastic Bridges of Linear Systems , 2014, IEEE Transactions on Automatic Control.

[26]  E. Yaz Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[27]  F. Ricci,et al.  Low Temperature and Gravitation Wave Detectors , 2014 .

[28]  M. Pavon,et al.  On the Markov processes of Schrödinger, the Feynman-Kac formula and stochastic control , 1990 .

[29]  Claudio Altafini,et al.  Modeling and Control of Quantum Systems: An Introduction , 2012, IEEE Transactions on Automatic Control.

[30]  Manfred Morari,et al.  The Linear Quadratic Regulator with chance constraints , 2013, 2013 European Control Conference (ECC).

[31]  M. Pavon,et al.  Lagrange lemma and the optimal control of diffusion. 1. Differentiable multipliers , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[32]  Domenico D'Alessandro,et al.  Optimal control of two-level quantum systems , 2001, IEEE Trans. Autom. Control..

[33]  Robert Skelton,et al.  A covariance control theory , 1985, 1985 24th IEEE Conference on Decision and Control.

[34]  T. Georgiou,et al.  Positive contraction mappings for classical and quantum Schrödinger systems , 2014, 1405.6650.

[35]  Tryphon T. Georgiou,et al.  The structure of state covariances and its relation to the power spectrum of the input , 2002, IEEE Trans. Autom. Control..

[36]  Tryphon T. Georgiou,et al.  Fast cooling for a system of stochastic oscillators , 2014, ArXiv.

[37]  Hans Föllmer,et al.  Random fields and diffusion processes , 1988 .

[38]  G. Prodi,et al.  Nonequilibrium steady-state fluctuations in actively cooled resonators. , 2009, Physical review letters.

[39]  Ian R. Petersen,et al.  State Distributions and Minimum Relative Entropy Noise Sequences in Uncertain Stochastic Systems: The Discrete-Time Case , 2012, SIAM J. Control. Optim..

[40]  Tryphon T. Georgiou,et al.  Optimal Steering of a Linear Stochastic System to a Final Probability Distribution, Part II , 2014, IEEE Transactions on Automatic Control.

[41]  K. A. Connors The Free Energy , 2003 .

[42]  G. Fredrickson The theory of polymer dynamics , 1996 .

[43]  M. Hautus A simple proof of Heymann's lemma , 1977 .

[44]  Tryphon T. Georgiou,et al.  Optimal Steering of a Linear Stochastic System to a Final Probability Distribution, Part I , 2016, IEEE Transactions on Automatic Control.

[45]  Roger W. Brockett,et al.  Notes on the Control of the Liouville Equation , 2012 .

[46]  Karolos M. Grigoriadis,et al.  Minimum-energy covariance controllers , 1997, Autom..

[47]  Michele Pavon,et al.  Lagrange lemma and the optimal control of diffusions II: nonlinear Lagrange functionals , 1995 .

[48]  Tryphon T. Georgiou,et al.  Entropic and Displacement Interpolation: A Computational Approach Using the Hilbert Metric , 2015, SIAM J. Appl. Math..

[49]  Christian L'eonard,et al.  O C ] 1 1 N ov 2 01 0 FROM THE SCHRÖDINGER PROBLEM TO THE MONGE-KANTOROVICH , 2010 .

[50]  R. Stephenson A and V , 1962, The British journal of ophthalmology.

[51]  M. Pavon,et al.  On Free Energy, Stochastic Control, and Schrödinger Processes , 1991 .

[52]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[53]  Ian R. Petersen,et al.  Minimum Relative Entropy State Transitions in Linear Stochastic Systems: the Continuous Time Case , 2012, ArXiv.

[54]  B. Jamison The Markov processes of Schrödinger , 1975 .

[55]  J. Willems Least squares stationary optimal control and the algebraic Riccati equation , 1971 .

[56]  T. Mikami Monge’s problem with a quadratic cost by the zero-noise limit of h-path processes , 2004 .

[57]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[58]  Abhishek Halder,et al.  Finite horizon linear quadratic Gaussian density regulator with Wasserstein terminal cost , 2016, 2016 American Control Conference (ACC).

[59]  Christian L'eonard A survey of the Schr\"odinger problem and some of its connections with optimal transport , 2013, 1308.0215.

[60]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[61]  Mark A. Hayes,et al.  Active Control of Dynamic Supraparticle Structures in Microchannels , 2001 .

[62]  Anton Wakolbinger A simplified variational characterization of Schrödinger processes , 1989 .

[63]  Efstathios Bakolas Optimal covariance control for stochastic linear systems subject to integral quadratic state constraints , 2016, 2016 American Control Conference (ACC).

[64]  D.R. Sahoo,et al.  Observer based imaging methods for Atomic Force Microscopy , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[65]  Paolo Dai Pra,et al.  A stochastic control approach to reciprocal diffusion processes , 1991 .

[66]  Alessandro Beghi,et al.  Continuous-time Gauss-Markov processes with fixed reciprocal dynamics , 1997 .

[67]  Murti V. Salapaka,et al.  Thermally driven non-contact atomic force microscopy , 2005 .

[68]  Tryphon T. Georgiou,et al.  Spectral analysis based on the state covariance: the maximum entropy spectrum and linear fractional parametrization , 2002, IEEE Trans. Autom. Control..

[69]  A. Krener,et al.  Dynamics and kinematics of reciprocal diffusions , 1993 .