Inference in finite state space non parametric Hidden Markov Models and applications

Hidden Markov models (HMMs) are intensively used in various fields to model and classify data observed along a line (e.g. time). The fit of such models strongly relies on the choice of emission distributions that are most often chosen among some parametric family. In this paper, we prove that finite state space non parametric HMMs are identifiable as soon as the transition matrix of the latent Markov chain has full rank and the emission probability distributions are linearly independent. This general result allows the use of semi- or non-parametric emission distributions. Based on this result we present a series of classification problems that can be tackled out of the strict parametric framework. We derive the corresponding inference algorithms. We also illustrate their use on few biological examples, showing that they may improve the classification performances.

[1]  Judith Rousseau,et al.  Nonparametric finite translation hidden Markov models and extensions , 2016 .

[2]  P. Rousseeuw,et al.  Wiley Series in Probability and Mathematical Statistics , 2005 .

[3]  Farzin Mokhtarian,et al.  A Non-Parametric HMM Learning Method for Shape Dynamics with Application to Human Motion Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[4]  L. Bordes,et al.  SEMIPARAMETRIC ESTIMATION OF A TWO-COMPONENT MIXTURE MODEL , 2006, math/0607812.

[5]  Thierry Dumont,et al.  Nonparametric regression on hidden phi-mixing variables: identifiability and consistency of a pseudo-likelihood based estimation procedure , 2012, 1209.0633.

[6]  Matthew Roughan,et al.  Hidden Markov model identifiability via tensors , 2013, 2013 IEEE International Symposium on Information Theory.

[7]  Laurent Bordes,et al.  Semiparametric Estimation of a Two-component Mixture Model where One Component is known , 2006 .

[8]  Dean Alderucci.  A SPECTRAL ALGORITHM FOR LEARNING HIDDEN MARKOV MODELS THAT HAVE SILENT STATES , 2015 .

[9]  Lifeng Shang,et al.  Nonparametric discriminant HMM and application to facial expression recognition , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  M. Lefebvre.  Applied probability and statistics , 2006 .

[11]  Xiao-Hua Zhou,et al.  NONPARAMETRIC ESTIMATION OF COMPONENT DISTRIBUTIONS IN A MULTIVARIATE MIXTURE , 2003 .

[12]  S. Geer.  Empirical Processes in M-Estimation , 2000 .

[13]  Fabrice Lefèvre,et al.  Non-parametric probability estimation for HMM-based automatic speech recognition , 2003, Comput. Speech Lang..

[14]  Dipankar Bandyopadhyay,et al.  Hidden Markov models for zero‐inflated Poisson counts with an application to substance use , 2011, Statistics in medicine.

[15]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[16]  Cathy Maugis,et al.  A non asymptotic penalized criterion for Gaussian mixture model selection , 2011 .

[17]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[18]  Eric Moulines,et al.  Inference in Hidden Markov Models (Springer Series in Statistics) , 2005 .

[19]  C. Matias,et al.  Identifiability of parameters in latent structure models with many observed variables , 2008, 0809.5032.

[20]  Stéphane Robin,et al.  Least-squares estimation of a convex discrete distribution , 2013, Comput. Stat. Data Anal..

[21]  E. Vernet.  Posterior consistency for nonparametric hidden Markov models with finite state space , 2013, 1311.3092.

[22]  Tao Jiang,et al.  The Regularized EM Algorithm , 2005, AAAI.

[23]  David R. Hunter,et al.  An EM-Like Algorithm for Semi- and Nonparametric Estimation in Multivariate Mixtures , 2009 .

[24]  Stéphane Robin,et al.  Unsupervised Classification for Tiling Arrays: ChIP-chip and Transcriptome , 2011, Statistical applications in genetics and molecular biology.

[25]  Tsung-I Lin,et al.  Finite mixture modelling using the skew normal distribution , 2007 .

[26]  Gesine Reinert,et al.  The Power of Detecting Enriched Patterns: An HMM Approach , 2010, J. Comput. Biol..

[27]  M. Shum,et al.  Identiability and Inference of Hidden Markov Models , 2013 .

[28]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[29]  M. Shum,et al.  Identifiability and Inference of Hidden Markov Models , 2013 .

[30]  Mark Gerstein,et al.  Bioinformatics Original Paper a Supervised Hidden Markov Model Framework for Efficiently Segmenting Tiling Array Data in Transcriptional and Chip-chip Experiments: Systematically Incorporating Validated Biological Knowledge , 2022 .

[31]  Stéphane Robin,et al.  Hidden Markov Models with mixtures as emission distributions , 2012, Statistics and Computing.

[32]  D. Hunter,et al.  Maximum smoothed likelihood for multivariate mixtures , 2011 .

[33]  T Petrie,et al.  Probabilistic functions of finite-state markov chains. , 1967, Proceedings of the National Academy of Sciences of the United States of America.

[34]  I. Johnstone,et al.  Density estimation by wavelet thresholding , 1996 .

[35]  Cristina Butucea,et al.  Semiparametric Mixtures of Symmetric Distributions , 2011, 1111.2247.

[36]  Martin F. Lambert,et al.  A non-parametric hidden Markov model for climate state identification , 2003 .

[37]  Laurent Couvreur,et al.  Wavelet-based non-parametric HMM's: theory and applications , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[38]  Gilles Celeux,et al.  Combining Mixture Components for Clustering , 2010, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[39]  S. Yakowitz,et al.  On the Identifiability of Finite Mixtures , 1968 .

[40]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[41]  Madalina Olteanu,et al.  Hidden Markov models for time series of counts with excess zeros , 2012, ESANN.