A regression model with a hidden logistic process for feature extraction from time series

A new approach for feature extraction from time series is proposed in this paper. This approach consists of a specific regression model incorporating a discrete hidden logistic process. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. The parameters of the hidden logistic process, in the inner loop of the EM algorithm, are estimated using a multi-class Iterative Reweighted Least-Squares (IRLS) algorithm. A piecewise regression algorithm and its iterative variant have also been considered for comparisons. An experimental study using simulated and real data reveals good performances of the proposed approach.

[1]  Yulia Kempner,et al.  Application of piece-wise regression to detecting internal structure of signal , 1992, Pattern Recognit..

[2]  J. B. Ramsey,et al.  Estimating Mixtures of Normal Distributions and Switching Regressions , 1978 .

[3]  P. Green.  Iteratively reweighted least squares for maximum likelihood estimation , 1984 .

[4]  Ke Chen,et al.  Improved learning algorithms for mixture of experts in multiclass classification , 1999, Neural Networks.

[5]  Marco Muselli,et al.  A New Learning Method for Piecewise Linear Regression , 2002, ICANN.

[6]  Lawrence R. Rabiner,et al.  A tutorial on Hidden Markov Models , 1986 .

[7]  M. Fridman.  Hidden Markov model regression , 1993 .

[8]  Walter D. Fisher.  On Grouping for Maximum Homogeneity , 1958 .

[9]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[10]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[11]  Lawrence Carin,et al.  Sparse multinomial logistic regression: fast algorithms and generalization bounds , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Steven R. Waterhouse,et al.  Classification and Regression using Mixtures of Experts , 1997 .

[13]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[14]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[15]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[16]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .