Beyond normality: Learning sparse probabilistic graphical models in the non-Gaussian setting

We present an algorithm to identify sparse dependence structure in continuous and non-Gaussian probability distributions, given a corresponding set of data. The conditional independence structure of an arbitrary distribution can be represented as an undirected graph (or Markov random field), but most algorithms for learning this structure are restricted to the discrete or Gaussian cases. Our new approach allows for more realistic and accurate descriptions of the distribution in question, and in turn better estimates of its sparse Markov structure. Sparsity in the graph is of interest as it can accelerate inference, improve sampling methods, and reveal important dependencies between variables. The algorithm relies on exploiting the connection between the sparsity of the graph and the sparsity of transport maps, which deterministically couple one probability measure to another.

[1]  H. Knothe Contributions to the theory of convex bodies. , 1957 .

[2]  Youssef M. Marzouk,et al.  A Multiscale Strategy for Bayesian Inference Using Transport Maps , 2015, SIAM/ASA J. Uncertain. Quantification.

[3]  Andrey G. Cherstvy,et al.  Anomalous, non-Gaussian tracer diffusion in crowded two-dimensional environments , 2015, 1508.02029.

[4]  Weidong Liu,et al.  Adaptive Thresholding for Sparse Covariance Matrix Estimation , 2011, 1102.2237.

[5]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[6]  Po-Ling Loh,et al.  Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses , 2012, NIPS.

[7]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[8]  Richard A. Frey,et al.  Predictive Inference for Big, Spatial, Non‐Gaussian Data: MODIS Cloud Data and its Change‐of‐Support , 2016 .

[9]  Philip Sura,et al.  Climatology of Non-Gaussian Atmospheric Statistics , 2013 .

[10]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[11]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[12]  G. Oehlert A note on the delta method , 1992 .

[13]  N. Shephard,et al.  Stochastic Volatility: Likelihood Inference And Comparison With Arch Models , 1996 .

[14]  Youssef M. Marzouk,et al.  Inference via Low-Dimensional Couplings , 2017, J. Mach. Learn. Res..

[15]  Youssef M. Marzouk,et al.  Bayesian inference with optimal maps , 2011, J. Comput. Phys..

[16]  C. Villani Optimal Transport: Old and New , 2008 .

[17]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[18]  V. Bogachev,et al.  Triangular transformations of measures , 2005 .

[19]  Youssef Marzouk,et al.  Transport Map Accelerated Markov Chain Monte Carlo , 2014, SIAM/ASA J. Uncertain. Quantification.

[20]  M. Moseley,et al.  Magnetic Resonance in Medicine 51:924–937 (2004) Characterizing Non-Gaussian Diffusion by Using Generalized Diffusion Tensors , 2022 .

[21]  Larry A. Wasserman,et al.  The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs , 2009, J. Mach. Learn. Res..

[22]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[23]  Jeffrey M. Hausdorff,et al.  Long-range anticorrelations and non-Gaussian behavior of the heartbeat. , 1993, Physical review letters.

[24]  M. Rosenblatt Remarks on a Multivariate Transformation , 1952 .

[25]  Larry A. Wasserman,et al.  High Dimensional Semiparametric Gaussian Copula Graphical Models. , 2012, ICML 2012.