Topology Selection in Graphical Models of Autoregressive Processes

An algorithm is presented for topology selection in graphical models of autoregressive Gaussian time series. The graph topology of the model represents the sparsity pattern of the inverse spectrum of the time series and characterizes conditional independence relations between the variables. The method proposed in the paper is based on an l1-type nonsmooth regularization of the conditional maximum likelihood estimation problem. We show that this reduces to a convex optimization problem and describe a large-scale algorithm that solves the dual problem via the gradient projection method. Results of experiments with randomly generated and real data sets are also included.

[1]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[2]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[3]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[4]  Stephen Gould,et al.  Projected Subgradient Methods for Learning Sparse Gaussians , 2008, UAI.

[5]  Karim Lounici.  Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators , 2008, 0801.4610.

[6]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[7]  Katya Scheinberg,et al.  IBM Research Report SINCO - A Greedy Coordinate Ascent Method for Sparse Inverse Covariance Selection Problem , 2009 .

[8]  D. Bessler,et al.  The Structure of Interdependence in International Stock Markets , 2003 .

[9]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[10]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[11]  E. Bullmore,et al.  Undirected graphs of frequency-dependent functional connectivity in whole brain networks , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[12]  Rainer Dahlhaus,et al.  Identification of synaptic connections in neural ensembles by graphical models , 1997, Journal of Neuroscience Methods.

[13]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[14]  Yong Zhang,et al.  An augmented Lagrangian approach for sparse principal component analysis , 2009, Mathematical Programming.

[15]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[16]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[17]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[18]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[19]  Rainer Dahlhaus,et al.  Partial correlation analysis for the identification of synaptic connections , 2003, Biological Cybernetics.

[20]  Michael I. Jordan,et al.  Learning graphical models for stationary time series , 2004, IEEE Transactions on Signal Processing.

[21]  Petre Stoica,et al.  Introduction to spectral analysis , 1997 .

[22]  Stephen P. Boyd,et al.  Determinant Maximization with Linear Matrix Inequality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[23]  R. Dahlhaus,et al.  Using Interaction Graphs for Analysing the Therapy Process , 2005, Psychotherapy and Psychosomatics.

[24]  O. SIAMJ..  SMOOTH OPTIMIZATION APPROACH FOR SPARSE COVARIANCE SELECTION∗ , 2009 .

[25]  J. M. Martínez,et al.  Inexact spectral projected gradient methods on convex sets , 2003 .

[26]  M. Wainwright,et al.  HIGH-DIMENSIONAL COVARIANCE ESTIMATION BY MINIMIZING l1-PENALIZED LOG-DETERMINANT DIVERGENCE BY PRADEEP RAVIKUMAR , 2009 .

[27]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[28]  R. Dahlhaus.  Graphical interaction models for multivariate time series1 , 2000 .

[29]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[30]  P. Zhao,et al.  The composite absolute penalties family for grouped and hierarchical variable selection , 2009, 0909.0411.

[31]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[32]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[33]  Zhaosong Lu,et al.  Adaptive First-Order Methods for General Sparse Inverse Covariance Selection , 2009, SIAM J. Matrix Anal. Appl..

[34]  Ursula Gather,et al.  Graphical models for multivariate time series from intensive care monitoring , 2002, Statistics in medicine.

[35]  Roland Fried,et al.  Decomposability and selection of graphical models for multivariate time series , 2003 .

[36]  V. Roychowdhury,et al.  Maximum-likelihood estimation of multivariate normal graphical models : large-scale numerical implementation and topology selection , 2022 .

[37]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[38]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[39]  Jens Timmer,et al.  Cross-spectral Analysis of tremor Time Series , 2000, Int. J. Bifurc. Chaos.

[40]  Lieven Vandenberghe,et al.  1 Graphical models of autoregressive processes , 2009 .

[41]  John G. Proakis,et al.  Digital Communications , 1983 .

[42]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[43]  Kim-Chuan Toh,et al.  Primal-Dual Path-Following Algorithms for Determinant Maximization Problems With Linear Matrix Inequalities , 1999, Comput. Optim. Appl..

[44]  Jianhua Z. Huang,et al.  Covariance matrix selection and estimation via penalised normal likelihood , 2006 .

[45]  Tom M. Mitchell,et al.  Learning to Decode Cognitive States from Brain Images , 2004, Machine Learning.

[46]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[47]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[48]  Michael Eichler,et al.  Fitting Graphical Interaction Models to Multivariate Time Series , 2006, UAI.

[49]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[50]  L. Vandenberghe,et al.  Convex Optimization in Signal Processing and Communications: Graphical models of autoregressive processes , 2009 .

[51]  Michael Eichler,et al.  Testing nonparametric and semiparametric hypotheses in vector stationary processes , 2008 .

[52]  D. Brillinger.  Time series - data analysis and theory , 1981, Classics in applied mathematics.

[53]  Bhuvana Ramabhadran,et al.  Map approach to learning sparse Gaussian Markov networks , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.