Alternating direction optimization algorithms for covariance completion problems

Second-order statistics of nonlinear dynamical systems can be obtained from experiments or numerical simulations. These statistics are relevant in understanding the fundamental physics, e.g., of fluid flows, and are useful for developing low-complexity models. Such models can be used for the purpose of control design and analysis. In many applications, only certain second-order statistics of a limited number of states are available. Thus, it is of interest to complete partially specified covariance matrices in a way that is consistent with the linearized dynamics. The dynamics impose structural constraints on admissible forcing correlations and state statistics. Solutions to such completion problems can be used to obtain stochastically driven linearized models. Herein, we address the covariance completion problem. We introduce an optimization criterion that combines the nuclear norm together with an entropy functional. The two, together, provide a numerically stable and scalable computational approach which is aimed at low complexity structures for stochastic forcing that can account for the observed statistics. We develop customized algorithms based on alternating direction methods that are well-suited for large scale problems.

[1]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[2]  Tryphon T. Georgiou,et al.  Spectral analysis based on the state covariance: the maximum entropy spectrum and linear fractional parametrization , 2002, IEEE Trans. Autom. Control..

[3]  M. R. Jovanović,et al.  Completion of partially known turbulent flow statistics via convex optimization , 2014 .

[4]  Tryphon T. Georgiou,et al.  Completion of partially known turbulent flow statistics , 2014, 2014 American Control Conference.

[5]  M. Dahleh,et al.  Energy amplification in channel flows with stochastic excitation , 2001 .

[6]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[7]  Fu Lin,et al.  Design of Optimal Sparse Feedback Gains via the Alternating Direction Method of Multipliers , 2011, IEEE Transactions on Automatic Control.

[8]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[9]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[10]  Richard G. Baraniuk,et al.  Fast Alternating Direction Optimization Methods , 2014, SIAM J. Imaging Sci..

[11]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[12]  Tryphon T. Georgiou,et al.  The structure of state covariances and its relation to the power spectrum of the input , 2002, IEEE Trans. Autom. Control..

[13]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[14]  P. Ioannou,et al.  Stochastic forcing of the linearized Navier–Stokes equations , 1993 .

[15]  B. Bamieh,et al.  Modeling flow statistics using the linearized Navier-Stokes equations , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[16]  Tryphon T. Georgiou,et al.  An ADMM algorithm for matrix completion of partially known state covariances , 2013, 52nd IEEE Conference on Decision and Control.

[17]  Ruslan Salakhutdinov,et al.  Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm , 2010, NIPS.

[18]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[19]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, ISIT.

[20]  Emmanuel J. Candès,et al.  Adaptive Restart for Accelerated Gradient Schemes , 2012, Foundations of Computational Mathematics.

[21]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[22]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[23]  Tryphon T. Georgiou,et al.  State covariances and the matrix completion problem , 2013, 52nd IEEE Conference on Decision and Control.

[24]  M. Jovanović,et al.  Low-complexity modeling of partially available second-order statistics via matrix completion , 2014 .

[25]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[26]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[27]  Xiaoming Yuan,et al.  Alternating Direction Method for Covariance Selection Models , 2011, Journal of Scientific Computing.

[28]  Bala Rajaratnam,et al.  Sparse Gaussian graphical model estimation via alternating minimization , 2014, 1405.3034.

[29]  Rashad Moarref,et al.  Model-based design of transverse wall oscillations for turbulent drag reduction , 2012, Journal of Fluid Mechanics.

[30]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[31]  Bassam Bamieh,et al.  Componentwise energy amplification in channel flows , 2005, Journal of Fluid Mechanics.