Robust Estimation of Latent Tree Graphical Models: Inferring Hidden States With Inexact Parameters

Latent tree graphical models are widely used in computational biology, signal and image processing, and network tomography. Here, we design a new efficient, estimation procedure for latent tree models, including Gaussian and discrete, reversible models, that significantly improves on previous sample requirement bounds. Our techniques are based on a new hidden state estimator that is robust to inaccuracies in estimated parameters. More precisely, we prove that latent tree models can be estimated with high probability in the so-called Kesten-Stigum regime with O(log2n) samples, where n is the number of nodes.

[1]  Sagi Snir,et al.  Fast and reliable reconstruction of phylogenetic trees with very short edges , 2008, SODA '08.

[2]  Vincent Y. F. Tan,et al.  Learning Gaussian Tree Models: Analysis of Error Exponents and Extremal Structures , 2009, IEEE Transactions on Signal Processing.

[3]  Vincent Y. F. Tan,et al.  Learning High-Dimensional Markov Forest Distributions: Analysis of Error Rates , 2010, J. Mach. Learn. Res..

[4]  Vincent Y. F. Tan,et al.  High-dimensional structure estimation in Ising models: Local separation criterion , 2011, 1107.1736.

[5]  Thomas G. Dietterich.  Adaptive computation and machine learning , 1998 .

[6]  Elchanan Mossel,et al.  Phylogenies without Branch Bounds: Contracting the Short, Pruning the Deep , 2008, SIAM J. Discret. Math..

[7]  Yuval Peres,et al.  Reconstruction on Trees: Exponential Moment Bounds for Linear Estimators , 2009, ArXiv.

[8]  Elchanan Mossel,et al.  Evolutionary trees and the Ising model on the Bethe lattice: a proof of Steel’s conjecture , 2005, ArXiv.

[9]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[10]  Elchanan Mossel.  Reconstruction on Trees: Beating the Second Eigenvalue , 2001 .

[11]  Elchanan Mossel,et al.  On the Inference of Large Phylogenies with Long Branches: How Long Is Too Long? , 2010, Bulletin of mathematical biology.

[12]  Martin J. Wainwright,et al.  Information-theoretic bounds on model selection for Gaussian Markov random fields , 2010, 2010 IEEE International Symposium on Information Theory.

[13]  Elchanan Mossel,et al.  Information flow on trees , 2001, math/0107033.

[14]  Avinatan Hassidim,et al.  Topology discovery of sparse random graphs with few participants , 2011, SIGMETRICS '11.

[15]  A. Willsky.  Multiresolution Markov models for signal and image processing , 2002, Proc. IEEE.

[16]  Verzekeren Naar Sparen,et al.  Cambridge , 1969, Humphrey Burton: In My Own Time.

[17]  R. Durrett.  Probability: Theory and Examples , 1993 .

[18]  Martin J. Wainwright,et al.  Information-Theoretic Limits of Selecting Binary Graphical Models in High Dimensions , 2009, IEEE Transactions on Information Theory.

[19]  Olgica Milenkovic,et al.  On modeling gene regulatory networks using Markov random fields , 2009, 2009 IEEE Information Theory Workshop on Networking and Information Theory.

[20]  T. W. Anderson,et al.  An Introduction to Multivariate Statistical Analysis , 1959 .

[21]  J. Lafferty,et al.  High-dimensional Ising model selection using ℓ1-regularized logistic regression , 2010, 1010.0311.

[22]  J. Schmee.  An Introduction to Multivariate Statistical Analysis , 1986 .

[23]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[24]  Y. Peres,et al.  Broadcasting on trees and the Ising model , 2000 .

[25]  Elchanan Mossel,et al.  On the Impossibility of Reconstructing Ancestral Data and Phylogenies , 2003, J. Comput. Biol..

[26]  Linda C. van der Gaag,et al.  Probabilistic Graphical Models , 2014, Lecture Notes in Computer Science.

[27]  Elchanan Mossel.  Distorted Metrics on Trees and Phylogenetic Forests , 2007, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[28]  Anima Anandkumar,et al.  Learning Loopy Graphical Models with Latent Variables: Efficient Methods and Guarantees , 2012, The Annals of Statistics.

[29]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[30]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[31]  Ram Rajagopal,et al.  Network delay inference from additive metrics , 2006, Random Struct. Algorithms.

[32]  Elchanan Mossel,et al.  Reconstruction of Markov Random Fields from Samples: Some Observations and Algorithms , 2007, SIAM J. Comput..

[33]  Lang Tong,et al.  A Large-Deviation Analysis of the Maximum-Likelihood Learning of Markov Tree Structures , 2009, IEEE Transactions on Information Theory.

[34]  Kevin Atteson,et al.  The Performance of Neighbor-Joining Methods of Phylogenetic Reconstruction , 1999, Algorithmica.

[35]  Robert Nowak,et al.  Network Tomography: Recent Developments , 2004 .

[36]  Allan Sly,et al.  Reconstruction for the Potts model , 2009, STOC '09.

[37]  Vincent Y. F. Tan,et al.  Learning Latent Tree Graphical Models , 2010, J. Mach. Learn. Res..

[38]  Elchanan Mossel,et al.  Learning nonsingular phylogenies and hidden Markov models , 2005, STOC '05.

[39]  H. Kesten,et al.  Additional Limit Theorems for Indecomposable Multidimensional Galton-Watson Processes , 1966 .

[40]  Tandy J. Warnow,et al.  A few logs suffice to build (almost) all trees (I) , 1999, Random Struct. Algorithms.

[41]  Elchanan Mossel.  Phase transitions in phylogeny , 2003, Transactions of the American Mathematical Society.

[42]  Tandy J. Warnow,et al.  A Few Logs Suffice to Build (almost) All Trees: Part II , 1999, Theor. Comput. Sci..

[43]  P. Erdös,et al.  A few logs suffice to build (almost) all trees (l): part I , 1997 .

[44]  S. Roch.  Toward Extracting All Phylogenetic Information from Matrices of Evolutionary Distances , 2010, Science.

[45]  Elchanan Mossel,et al.  The Complexity of Distinguishing Markov Random Fields , 2008, APPROX-RANDOM.

[46]  Gene H. Golub,et al.  Matrix computations , 1983 .