Modeling of multivariate time series using hidden markov models
Abstract:Vector-valued (or multivariate) time series data commonly occur in various sciences. While modeling univariate time series is well-studied, modeling of multivariate time series, especially finite-valued or categorical, has been relatively unexplored. In this dissertation, we employ hidden Markov models (HMMs) to capture temporal and multivariate dependencies in the multivariate time series data. We modularize the process of building such models by separating the modeling of temporal dependence, multivariate dependence, and non-stationary behavior. We also propose new methods of modeling multivariate dependence for categorical and real-valued data while drawing parallels between these two seemingly different types of data. Since this work is in part motivated by the problem of prediction precipitation over geographic regions from the multiple weather stations, we present in detail models pertinent to this hydrological application and perform a thorough analysis of the models on data collected from a number of different geographic regions.
暂无分享,去 创建一个
[1] David M. Pennock,et al. Mixtures of Conditional Maximum Entropy Models , 2003, ICML.
[2] Michael I. Jordan,et al. Thin Junction Trees , 2001, NIPS.
[3] D. Wilks,et al. The weather generation game: a review of stochastic weather models , 1999 .
[4] Joshua Goodman,et al. Sequential Conditional Generalized Iterative Scaling , 2002, ACL.
[5] Dennis P. Lettenmaier,et al. A hierarchical stochastic model of large‐scale atmospheric circulation patterns and multiple station daily precipitation , 1992 .
[6] S. Chib. Calculating posterior distributions and modal estimates in Markov mixture models , 1996 .
[7] Ronald Rosenfeld,et al. Adaptive Statistical Language Modeling; A Maximum Entropy Approach , 1994 .
[8] A. B. Poritz,et al. Linear predictive hidden Markov models and the speech signal , 1982, ICASSP.
[9] Marina Meila,et al. An Accelerated Chow and Liu Algorithm: Fitting Tree Distributions to High-Dimensional Sparse Data , 1999, ICML.
[10] Daniel S. Wilks,et al. Realizations of Daily Weather in Forecast Seasonal Climate , 2002 .
[11] J. Ashford,et al. Multi-variate probit analysis. , 1970, Biometrics.
[12] Alex Pentland,et al. Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
[13] José Manuel Gutiérrez,et al. Bayesian Networks for Probabilistic Weather Prediction , 2002, ECAI.
[14] Michael I. Jordan,et al. Probabilistic Independence Networks for Hidden Markov Probability Models , 1997, Neural Computation.
[15] D. Wilks. Multisite generalization of a daily stochastic precipitation generation model , 1998 .
[16] D. Haussler,et al. Hidden Markov models in computational biology. Applications to protein modeling. , 1993, Journal of molecular biology.
[17] Andrew W. Moore,et al. Using Tarjan's Red Rule for Fast Dependency Tree Construction , 2002, NIPS.
[18] Upmanu Lall,et al. A k‐nearest‐neighbor simulator for daily precipitation and other weather variables , 1999 .
[19] Lain L. MacDonald,et al. Hidden Markov and Other Models for Discrete- valued Time Series , 1997 .
[20] Yoshua Bengio,et al. Markovian Models for Sequential Data , 2004 .
[21] Michael I. Jordan,et al. Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..
[22] L. Baum,et al. Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .
[23] David Maxwell Chickering,et al. A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.
[24] Alan Genz,et al. Numerical computation of rectangular bivariate and trivariate normal and t probabilities , 2004, Stat. Comput..
[25] Michael I. Jordan,et al. Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.
[26] Frederick Jelinek,et al. Statistical methods for speech recognition , 1997 .
[27] Peter Guttorp,et al. A Hidden Markov Model for Space‐Time Precipitation , 1991 .
[28] David T. Brown,et al. A Note on Approximations to Discrete Probability Distributions , 1959, Inf. Control..
[29] L. Baum,et al. A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .
[30] Tom Minka,et al. A family of algorithms for approximate Bayesian inference , 2001 .
[31] I. Csiszár. $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .
[32] Andrew J. Viterbi,et al. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.
[33] James D. Hamilton. Time Series Analysis , 1994 .
[34] James P. Hughes,et al. A hidden Markov model for downscaling synoptic atmospheric patterns to precipitation amounts , 2000 .
[35] Judea Pearl,et al. Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.
[36] Frank Jensen,et al. Optimal junction Trees , 1994, UAI.
[37] Lawrence R. Rabiner,et al. A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.
[38] Ross D. Shachter. Bayes-Ball: The Rational Pastime (for Determining Irrelevance and Requisite Information in Belief Networks and Influence Diagrams) , 1998, UAI.
[39] Andrew McCallum,et al. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.
[40] Yoshua Bengio,et al. An Input Output HMM Architecture , 1994, NIPS.
[41] G. McCabe,et al. Simulation of precipitation by weather type analysis , 1991 .
[42] P. Guttorp,et al. A non‐homogeneous hidden Markov model for precipitation occurrence , 1999 .
[43] David R. Cox,et al. A simple spatial-temporal model of rainfall , 1988, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.
[44] Padhraic Smyth,et al. Conditional Chow-Liu Tree Structures for Modeling Discrete-Valued Vector Time Series , 2004, UAI.
[45] C. N. Liu,et al. Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.
[46] James P. Hughes,et al. Incorporating Spatial Dependence and Atmospheric Data in a Model of Precipitation , 1994 .
[47] M. Do. Fast approximation of Kullback-Leibler distance for dependence trees and hidden Markov models , 2003, IEEE Signal Processing Letters.
[48] A. Bárdossy,et al. SPACE-TIME MODEL FOR DAILY RAINFALL USING ATMOSPHERIC CIRCULATION PATTERNS , 1992 .
[49] D. Rubin,et al. Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .
[50] Nathan Srebro,et al. Maximum likelihood bounded tree-width Markov networks , 2001, Artif. Intell..
[51] Adam L. Berger,et al. A Maximum Entropy Approach to Natural Language Processing , 1996, CL.
[52] J. N. R. Jeffers,et al. Graphical Models in Applied Multivariate Statistics. , 1990 .
[53] Frank Bretz,et al. Comparison of Methods for the Computation of Multivariate t Probabilities , 2002 .
[54] Nir Friedman,et al. Learning Belief Networks in the Presence of Missing Values and Hidden Variables , 1997, ICML.
[55] S. Chib,et al. Analysis of multivariate probit models , 1998 .
[56] S. L. Scott. Bayesian Methods for Hidden Markov Models , 2002 .
[57] David Heckerman,et al. Knowledge Representation and Inference in Similarity Networks and Bayesian Multinets , 1996, Artif. Intell..
[58] Thomas M. Cover,et al. Elements of Information Theory , 2005 .
[59] Emmanuel Lesaffre,et al. Existence and Uniqueness of the Maximum Likelihood Estimator for a Multivariate Probit Model , 1992 .
[60] A. Berchtold. The double chain markov model , 1999 .