Learning Quasi-Kronecker Product Graphical Models

We consider the problem of learning graphical models where the support of the concentration matrix can be decomposed as a Kronecker product. We propose a method that uses the Bayesian hierarchical learning modeling approach. Thanks to the particular structure of the graph, we use a the number of hyperparameters which is small compared to the number of nodes in the graphical model. In this way, we avoid overfitting in the estimation of the hyperparameters. Finally, we test the effectiveness of the proposed method by a numerical example.

[1]  Jianhua Z. Huang,et al.  Covariance matrix selection and estimation via penalised normal likelihood , 2006 .

[2]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[3]  Lieven Vandenberghe,et al.  Topology Selection in Graphical Models of Autoregressive Processes , 2010, J. Mach. Learn. Res..

[4]  Katya Scheinberg,et al.  Sparse Markov net learning with priors on regularization parameters , 2010, ISAIM.

[5]  Mattia Zorzi,et al.  Factor analysis of moving average processes , 2014, 2015 European Control Conference (ECC).

[6]  Alfred O. Hero,et al.  On Convergence of Kronecker Graphical Lasso Algorithms , 2012, IEEE Transactions on Signal Processing.

[7]  Bhuvana Ramabhadran,et al.  Map approach to learning sparse Gaussian Markov networks , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[9]  Jeff G. Schneider,et al.  Learning Multiple Tasks with a Sparse Matrix-Normal Penalty , 2010, NIPS.

[10]  P. Dutilleul.  The mle algorithm for the matrix normal distribution , 1999 .

[11]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[12]  Michael I. Jordan.  Graphical Models , 2003 .

[13]  Alfred O. Hero,et al.  Covariance Estimation in High Dimensions Via Kronecker Product Expansions , 2013, IEEE Transactions on Signal Processing.

[14]  Daniele Alpago,et al.  Identification of Sparse Reciprocal Graphical Models , 2018, IEEE Control Systems Letters.

[15]  Fetsje Bijma,et al.  The spatiotemporal MEG covariance matrix modeled as a sum of Kronecker products , 2005, NeuroImage.

[16]  T. Speed,et al.  Gaussian Markov Distributions over Finite Graphs , 1986 .

[17]  Petre Stoica,et al.  On Estimation of Covariance Matrices With Kronecker Product Structure , 2008, IEEE Transactions on Signal Processing.

[18]  Alexandre d'Aspremont,et al.  First-Order Methods for Sparse Covariance Selection , 2006, SIAM J. Matrix Anal. Appl..

[19]  Mattia Zorzi,et al.  Robust Identification of “Sparse Plus Low-rank” Graphical Models: An Optimization Approach , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[20]  Mattia Zorzi,et al.  AR Identification of Latent-Variable Graphical Models , 2014, IEEE Transactions on Automatic Control.

[21]  Zhenyu Zhou,et al.  Approximate maximum likelihood hyperparameter estimation for Gibbs priors , 1995, Proceedings., International Conference on Image Processing.

[22]  Genevera I. Allen,et al.  TRANSPOSABLE REGULARIZED COVARIANCE MODELS WITH AN APPLICATION TO MISSING DATA IMPUTATION. , 2009, The annals of applied statistics.

[23]  Michel Verhaegen,et al.  Kronecker-ARX models in identifying (2D) spatial-temporal systems , 2017 .

[24]  Mattia Zorzi,et al.  Factor analysis with finite data , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[25]  Mattia Zorzi,et al.  Factor Models With Real Data: A Robust Estimation of the Number of Factors , 2017, IEEE Transactions on Automatic Control.