A Nonconvex Framework for Structured Dynamic Covariance Recovery

We propose a flexible yet interpretable model for high-dimensional data with time-varying second order statistics, motivated and applied to functional neuroimaging data. Motivated by the neuroscience literature, we factorize the covariances into sparse spatial and smooth temporal components. While this factorization results in both parsimony and domain interpretability, the resulting estimation problem is nonconvex. To this end, we design a two-stage optimization scheme with a carefully tailored spectral initialization, combined with iteratively refined alternating projected gradient descent. We prove a linear convergence rate up to a nontrivial statistical error for the proposed descent scheme and establish sample complexity guarantees for the estimator. We further quantify the statistical error for the multivariate Gaussian case. Empirical results using simulated and real brain imaging data illustrate that our approach outperforms existing baselines.

[1]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[2]  A. B. Poritz,et al.  Linear predictive hidden Markov models and the speech signal , 1982, ICASSP.

[3]  Rui Li,et al.  Multivariate Sparse Coding of Nonstationary Covariances with Gaussian Processes , 2019, NeurIPS.

[4]  D. Giannone,et al.  Large Bayesian vector auto regressions , 2010 .

[5]  Martin J. Wainwright,et al.  Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees , 2015, ArXiv.

[6]  Madeleine Udell,et al.  Why Are Big Data Matrices Approximately Low Rank? , 2017, SIAM J. Math. Data Sci..

[7]  Tuo Zhao,et al.  Stochastic Variance Reduced Optimization for Nonconvex Sparse Learning , 2016, ICML.

[8]  Roman Filipovych,et al.  Sparse Dictionary Learning of Resting State fMRI Networks , 2012, 2012 Second International Workshop on Pattern Recognition in NeuroImaging.

[9]  Han Liu,et al.  Joint estimation of multiple graphical models from high dimensional time series , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[10]  Max Simchowitz,et al.  Low-rank Solutions of Linear Matrix Equations via Procrustes Flow , 2015, ICML.

[11]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[12]  M. Posner,et al.  Localization of cognitive operations in the human brain. , 1988, Science.

[13]  M. Kolar,et al.  Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach , 2018, Electronic Journal of Statistics.

[14]  Xinghao Qiao,et al.  Doubly functional graphical models in high dimensions , 2020, Biometrika.

[15]  A. Skripnikov,et al.  Regularized joint estimation of related vector autoregressive models , 2019, Comput. Stat. Data Anal..

[16]  Stephen M. Smith,et al.  Brain network dynamics are hierarchically organized in time , 2017, Proceedings of the National Academy of Sciences.

[17]  Kent A. Kiehl,et al.  A method for evaluating dynamic functional network connectivity and task-modulation: application to schizophrenia , 2010, Magnetic Resonance Materials in Physics, Biology and Medicine.

[18]  Renato D. C. Monteiro,et al.  Digital Object Identifier (DOI) 10.1007/s10107-004-0564-1 , 2004 .

[19]  David B. Dunson,et al.  Bayesian nonparametric covariance regression , 2011, J. Mach. Learn. Res..

[20]  Gregor Kastner,et al.  Efficient Bayesian Inference for Multivariate Factor Stochastic Volatility Models , 2016, 1602.08154.

[21]  Christopher J. Paciorek,et al.  Nonstationary Gaussian Processes for Regression and Spatial Modelling , 2003 .

[22]  M. Fox,et al.  Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging , 2007, Nature Reviews Neuroscience.

[23]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[24]  Moritz Hardt,et al.  Understanding Alternating Minimization for Matrix Completion , 2013, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[25]  Robert F. Engle,et al.  Large Dynamic Covariance Matrices , 2017 .

[26]  Emily B Fox,et al.  Statistical model-based approaches for functional connectivity analysis of neuroimaging data , 2019, Current Opinion in Neurobiology.

[27]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[28]  T. Ge,et al.  Resting brain dynamics at different timescales capture distinct aspects of human behavior , 2019, Nature Communications.

[29]  Dimitri Van De Ville,et al.  The dynamic functional connectome: State-of-the-art and perspectives , 2017, NeuroImage.

[30]  Herman Rubin,et al.  Statistical Inference in Factor Analysis , 1956 .

[31]  Ole Winther,et al.  Bayesian Structure Learning for Dynamic Brain Connectivity , 2018, AISTATS.

[32]  V. Calhoun,et al.  The Chronnectome: Time-Varying Connectivity Networks as the Next Frontier in fMRI Data Discovery , 2014, Neuron.

[33]  Adam S. Charles,et al.  Learning Spatially-correlated Temporal Dictionaries for Calcium Imaging , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[35]  James D. B. Nelson,et al.  Regularized Estimation of Piecewise Constant Gaussian Graphical Models: The Group-Fused Graphical Lasso , 2015, 1512.06171.

[36]  Dimitri Van De Ville,et al.  On spurious and real fluctuations of dynamic functional connectivity during rest , 2015, NeuroImage.

[37]  Krzysztof J. Gorgolewski,et al.  The Dynamics of Functional Brain Networks: Integrated Network States during Cognitive Task Performance , 2015, Neuron.

[38]  Edward T. Bullmore,et al.  On the use of correlation as a measure of network connectivity , 2012, NeuroImage.

[39]  M. Raydan,et al.  Alternating Projection Methods , 2011 .

[40]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[41]  R. Poldrack,et al.  Temporal metastates are associated with differential patterns of time-resolved connectivity, network topology, and attention , 2016, Proceedings of the National Academy of Sciences.

[42]  J. Berge,et al.  Orthogonal procrustes rotation for two or more matrices , 1977 .

[43]  John von Neumann,et al.  1. A Certain Zero-sum Two-person Game Equivalent to the Optimal Assignment Problem , 1953 .

[44]  Jörn Diedrichsen,et al.  A probabilistic MR atlas of the human cerebellum , 2009, NeuroImage.

[45]  Tengyao Wang,et al.  A useful variant of the Davis--Kahan theorem for statisticians , 2014, 1405.0680.

[46]  E. Fox,et al.  Identifiability and estimation of structural vector autoregressive models for subsampled and mixed-frequency time series. , 2017, Biometrika.

[47]  B. Minasny,et al.  The Matérn function as a general model for soil variograms , 2005 .

[48]  Oluwasanmi Koyejo,et al.  Human cognition involves the dynamic integration of neural activity and neuromodulatory systems , 2019, Nature Neuroscience.

[49]  Jian Zhang,et al.  Factorized estimation of high‐dimensional nonparametric covariance models , 2021, Scandinavian Journal of Statistics.

[50]  Hansheng Wang,et al.  Nonparametric Covariance Model , 2008, Statistica Sinica.

[51]  Timothy O. Laumann,et al.  Generation and Evaluation of a Cortical Area Parcellation from Resting-State Correlations. , 2016, Cerebral cortex.

[52]  J. Duyn,et al.  Investigation of Low Frequency Drift in fMRI Signal , 1999, NeuroImage.

[53]  Stephen P. Boyd,et al.  Network Inference via the Time-Varying Graphical Lasso , 2017, KDD.

[54]  N. Ayache,et al.  Log‐Euclidean metrics for fast and simple calculus on diffusion tensors , 2006, Magnetic resonance in medicine.

[55]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[56]  Le Song,et al.  Estimating time-varying networks , 2008, ISMB 2008.

[57]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[58]  Zhaoran Wang,et al.  Low-Rank and Sparse Structure Pursuit via Alternating Minimization , 2016, AISTATS.

[59]  M. Schölvinck,et al.  Tracking brain arousal fluctuations with fMRI , 2016, Proceedings of the National Academy of Sciences.

[60]  Richard A. Davis,et al.  Sparse Vector Autoregressive Modeling , 2012, 1207.0520.

[61]  Martin J. Wainwright,et al.  High-Dimensional Statistics , 2019 .

[62]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[63]  Anastasios Kyrillidis,et al.  Dropping Convexity for Faster Semi-definite Optimization , 2015, COLT.

[64]  D. F. Ahelegbey,et al.  Bayesian Graphical Models for Structural Vector Autoregressive Processes , 2012, Journal of Applied Econometrics.

[65]  Renato D. C. Monteiro,et al.  A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization , 2003, Math. Program..