Low rank continuous-space graphical models

Constructing tractable dependent probability distributions over structured continuous random vectors is a central problem in statistics and machine learning. It has proven difficult to find general constructions for models in which ecient exact inference is possible, outside of the classical cases of models with restricted graph structure (chain, tree, etc.) and linear-Gaussian or discrete potentials. In this work we identify a graphical model class in which exact inference can be performed efficiently, owing to a certain “low-rank” structure in the potentials. While we focus on the case of tree graphical models, the lowrank treatment can also be applied for ecient exact inference in certain sparsely-loopy models. We explore this new class of models by applying the resulting inference methods to neural spike rate estimation and motioncapture joint-angle smoothing tasks.

[1]  Arnaud Doucet,et al.  Generalized Polya Urn for Time-varying Dirichlet Process Mixtures , 2007, UAI.

[2]  Yee Whye Teh,et al.  Dependent Dirichlet Process Spike Sorting , 2008, NIPS.

[3]  Richard E. Turner,et al.  Probabilistic amplitude and frequency demodulation , 2011, NIPS.

[4]  Charles F. Cadieu,et al.  Phase Coupling Estimation from Multivariate Phase Statistics , 2009, Neural Computation.

[5]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[6]  Carlos Guestrin,et al.  Nonparametric Tree Graphical Models via Kernel Embeddings , 2010 .

[7]  Le Song,et al.  Hilbert Space Embeddings of Hidden Markov Models , 2010, ICML.

[8]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[9]  Jean-Paul Chilès,et al.  Wiley Series in Probability and Statistics , 2012 .

[10]  S. R. Jammalamadaka,et al.  Directional Statistics, I , 2011 .

[11]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[12]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[13]  Tom Minka,et al.  A family of algorithms for approximate Bayesian inference , 2001 .

[14]  M. Pitt,et al.  Constructing Stationary Time Series Models Using Auxiliary Variables With Applications , 2005 .

[15]  Stephen G. Walker,et al.  Constructing First Order Stationary Autoregressive Models via Latent Processes , 2002 .

[16]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[17]  Byron Boots,et al.  Reduced-Rank Hidden Markov Models , 2009, AISTATS.

[18]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[19]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .