Learning Sparse Graph with Minimax Concave Penalty under Gaussian Markov Random Fields

This paper presents a convex-analytic framework to learn sparse graphs from data. While our problem formulation is inspired by an extension of the graphical lasso using the so-called combinatorial graph Laplacian framework, a key difference is the use of a nonconvex alternative to the `1 norm to attain graphs with better interpretability. Specifically, we use the weakly-convex minimax concave penalty (the difference between the `1 norm and the Huber function) which is known to yield sparse solutions with lower estimation bias than `1 for regression problems. In our framework, the graph Laplacian is replaced in the optimization by a linear transform of the vector corresponding to its upper triangular part. Via a reformulation relying on Moreau’s decomposition, we show that overall convexity is guaranteed by introducing a quadratic function to our cost function. The problem can be solved efficiently by the primaldual splitting method, of which the admissible conditions for provable convergence are presented. Numerical examples show that the proposed method significantly outperforms the existing graph learning methods with reasonable CPU time. INDEX TERMS Graph signal processing, graph learning, graphical lasso, minimax concave penalty, primal-dual splitting method, proximity operator

[1]  Peter Congdon,et al.  Gaussian Markov Random Fields: Theory and Applications , 2007 .

[2]  Defeng Sun,et al.  Learning Graph Laplacian with MCP , 2020, ArXiv.

[3]  Trevor J. Hastie,et al.  The Graphical Lasso: New Insights and Alternatives , 2011, Electronic journal of statistics.

[4]  Masahiro Yukawa,et al.  Normalized Least-Mean-Square Algorithms with Minimax Concave Penalty , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Jugal K. Kalita,et al.  Network Anomaly Detection: Methods, Systems and Tools , 2014, IEEE Communications Surveys & Tutorials.

[6]  Isao Yamada,et al.  Linearly-involved Moreau-Enhanced-over-Subspace Model: Debiased Sparse Modeling and Stable Outlier-Robust Regression , 2022 .

[7]  Masahiro Yukawa,et al.  Distributed Sparse Optimization With Minimax Concave Regularization , 2021, 2021 IEEE Statistical Signal Processing Workshop (SSP).

[8]  Sandeep Kumar,et al.  A Unified Framework for Structured Graph Learning via Spectral Constraints , 2019, J. Mach. Learn. Res..

[9]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[10]  Santiago Segarra,et al.  Network Topology Inference from Spectral Templates , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[11]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[12]  Masahiro Yukawa,et al.  Supervised nonnegative matrix factorization via minimization of regularized Moreau-envelope of divergence function with application to music transcription , 2017, J. Frankl. Inst..

[13]  Ivan W. Selesnick,et al.  Sparse Regularization via Convex Analysis , 2017, IEEE Transactions on Signal Processing.

[14]  Daniel Pérez Palomar,et al.  Nonconvex Sparse Graph Learning under Laplacian Constrained Graphical Model , 2020, NeurIPS.

[15]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[16]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[17]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[18]  Hao Wang,et al.  Bayesian Graphical Lasso Models and Efficient Posterior Computation , 2012 .

[19]  Gaigai Cai,et al.  Nonconvex Sparse Regularization and Convex Optimization for Bearing Fault Diagnosis , 2018, IEEE Transactions on Industrial Electronics.

[20]  Cun-Hui Zhang Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[21]  Masahiro Yukawa,et al.  Robust Recovery of Jointly-Sparse Signals Using Minimax Concave Loss Function , 2021, IEEE Transactions on Signal Processing.

[22]  Shiliang Sun,et al.  Network-Scale Traffic Modeling and Forecasting with Graphical Lasso and Neural Networks , 2012 .

[23]  Nicolas Tremblay,et al.  Approximate Fast Graph Fourier Transforms via Multilayer Sparse Approximations , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[24]  Antonio Ortega,et al.  Graph Learning From Data Under Laplacian and Structural Constraints , 2016, IEEE Journal of Selected Topics in Signal Processing.

[25]  Pierre Vandergheynst,et al.  GSPBOX: A toolbox for signal processing on graphs , 2014, ArXiv.

[26]  Josiane Mothe,et al.  Nonconvex Regularizations for Feature Selection in Ranking With Sparse SVM , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Antonio Ortega,et al.  Time-varying Graph Learning Based on Sparseness of Temporal Variation , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Nir Friedman,et al.  Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning , 2009 .

[29]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[30]  E. A. Nurminskii The quasigradient method for the solving of the nonlinear programming problems , 1973 .

[31]  Xiaotong Shen,et al.  Journal of the American Statistical Association Likelihood-based Selection and Sharp Parameter Estimation Likelihood-based Selection and Sharp Parameter Estimation , 2022 .

[32]  Peilin Liu,et al.  A Survey on Nonconvex Regularization-Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning , 2018, IEEE Access.

[33]  O Mason,et al.  Graph theory and networks in Biology. , 2006, IET systems biology.

[34]  Joshua B. Tenenbaum,et al.  Discovering Structure by Learning Sparse Graphs , 2010 .

[35]  Antonio Ortega,et al.  A Graph Learning Algorithm Based On Gaussian Markov Random Fields And Minimax Concave Penalty , 2021, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Stephen P. Boyd,et al.  Network Inference via the Time-Varying Graphical Lasso , 2017, KDD.

[37]  T. Hastie,et al.  SparseNet: Coordinate Descent With Nonconvex Penalties , 2011, Journal of the American Statistical Association.

[38]  F. Vecchio,et al.  Connectome: Graph theory application in functional brain network architecture , 2017, Clinical neurophysiology practice.

[39]  Santiago Segarra,et al.  Connecting the Dots: Identifying Network Structure via Graph Signal Processing , 2018, IEEE Signal Processing Magazine.

[40]  Masao Yamagishi,et al.  Linearly Involved Generalized Moreau Enhanced Models and Their Proximal Splitting Algorithm under Overall Convexity Condition , 2019, ArXiv.

[41]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[42]  Pascal Frossard,et al.  Learning Graphs From Data: A Signal Representation Perspective , 2018, IEEE Signal Processing Magazine.

[43]  Jianqing Fan,et al.  Sparsistency and Rates of Convergence in Large Covariance Matrix Estimation. , 2007, Annals of statistics.