Discrete-Time Linear-Quadratic Regulation via Optimal Transport

In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for the optimal transport map in the case of linear-time varying systems is derived, along with an algorithm for computing the optimal map. Two numerical examples pertaining to swarm deployment demonstrate the practical applicability of the model, and performance of the numerical method.

[1]  Massimo Fornasier,et al.  Mean Field Control Hierarchy , 2016, Applied Mathematics & Optimization.

[2]  Tryphon T. Georgiou,et al.  Optimal Steering of a Linear Stochastic System to a Final Probability Distribution—Part III , 2014, IEEE Transactions on Automatic Control.

[3]  Hossein Mobahi,et al.  Learning with a Wasserstein Loss , 2015, NIPS.

[4]  Gabriel Peyré,et al.  Optimal Transport with Proximal Splitting , 2013, SIAM J. Imaging Sci..

[5]  Efstathios Bakolas,et al.  Finite-horizon covariance control for discrete-time stochastic linear systems subject to input constraints , 2018, Autom..

[6]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[7]  Spring Berman,et al.  Optimal Transport Over Deterministic Discrete-Time Nonlinear Systems Using Stochastic Feedback Laws , 2018, IEEE Control Systems Letters.

[8]  Behçet Açikmese,et al.  Decentralized probabilistic density control of autonomous swarms with safety constraints , 2015, Auton. Robots.

[9]  Yann Brenier,et al.  A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem , 2000, Numerische Mathematik.

[10]  Gabriel Peyré,et al.  The Numerical Tours of Signal Processing , 2011, Computing in Science & Engineering.

[11]  Behçet Açikmese,et al.  Optimal Mass Transport and Kernel Density Estimation for State-Dependent Networked Dynamic Systems , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[12]  Tryphon T. Georgiou,et al.  Optimal Transport Over a Linear Dynamical System , 2015, IEEE Transactions on Automatic Control.

[13]  Julien Rabin,et al.  Wasserstein Barycenter and Its Application to Texture Mixing , 2011, SSVM.

[14]  Behcet Acikmese,et al.  Velocity Field Generation for Density Control of Swarms using Heat Equation and Smoothing Kernels , 2017, 1710.00299.

[15]  Gabriel Peyré,et al.  Fast Dictionary Learning with a Smoothed Wasserstein Loss , 2016, AISTATS.

[16]  Jean-Baptiste Pomet,et al.  Mass Transportation with LQ Cost Functions , 2011, ArXiv.

[17]  Viet Anh Nguyen,et al.  Wasserstein Distributionally Robust Kalman Filtering , 2018, NeurIPS.

[18]  Panagiotis Tsiotras,et al.  Finite-horizon covariance control of linear time-varying systems , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[19]  Julien Rabin,et al.  Wasserstein regularization of imaging problem , 2011, 2011 18th IEEE International Conference on Image Processing.

[20]  Gabriel Peyré,et al.  Entropic Approximation of Wasserstein Gradient Flows , 2015, SIAM J. Imaging Sci..

[21]  Jason Altschuler,et al.  Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration , 2017, NIPS.

[22]  Daniel Kuhn,et al.  Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations , 2015, Mathematical Programming.

[23]  M. Knott,et al.  On the optimal mapping of distributions , 1984 .

[24]  A. Galichon,et al.  Optimal Transport Methods in Economics , 2016 .

[25]  Tryphon T. Georgiou,et al.  Optimal Steering of a Linear Stochastic System to a Final Probability Distribution, Part II , 2014, IEEE Transactions on Automatic Control.

[26]  Alessandro Rudi,et al.  Approximating the Quadratic Transportation Metric in Near-Linear Time , 2018, ArXiv.

[27]  Y. Brenier.  Polar Factorization and Monotone Rearrangement of Vector-Valued Functions , 1991 .

[28]  Tryphon T. Georgiou,et al.  Optimal Steering of a Linear Stochastic System to a Final Probability Distribution, Part I , 2016, IEEE Transactions on Automatic Control.

[29]  Viet Anh Nguyen,et al.  Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning , 2019, Operations Research & Management Science in the Age of Analytics.

[30]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[31]  Andrei Agrachev,et al.  Optimal transportation under nonholonomic constraints , 2007, 0710.0408.

[32]  Efstathios Bakolas,et al.  Stochastic linear systems subject to constraints , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[33]  C. Villani.  Optimal Transport: Old and New , 2008 .

[34]  Sonia Martínez,et al.  Distributed Optimal Transport for the Deployment of Swarms , 2018, 2018 IEEE Conference on Decision and Control (CDC).