相关论文

Modeling of multivariate time series using hidden markov models

Abstract:Vector-valued (or multivariate) time series data commonly occur in various sciences. While modeling univariate time series is well-studied, modeling of multivariate time series, especially finite-valued or categorical, has been relatively unexplored. In this dissertation, we employ hidden Markov models (HMMs) to capture temporal and multivariate dependencies in the multivariate time series data. We modularize the process of building such models by separating the modeling of temporal dependence, multivariate dependence, and non-stationary behavior. We also propose new methods of modeling multivariate dependence for categorical and real-valued data while drawing parallels between these two seemingly different types of data. Since this work is in part motivated by the problem of prediction precipitation over geographic regions from the multiple weather stations, we present in detail models pertinent to this hydrological application and perform a thorough analysis of the models on data collected from a number of different geographic regions.

参考文献

[1]  David M. Pennock,et al.  Mixtures of Conditional Maximum Entropy Models , 2003, ICML.

[2]  Michael I. Jordan,et al.  Thin Junction Trees , 2001, NIPS.

[3]  D. Wilks,et al.  The weather generation game: a review of stochastic weather models , 1999 .

[4]  Joshua Goodman,et al.  Sequential Conditional Generalized Iterative Scaling , 2002, ACL.

[5]  Dennis P. Lettenmaier,et al.  A hierarchical stochastic model of large‐scale atmospheric circulation patterns and multiple station daily precipitation , 1992 .

[6]  S. Chib Calculating posterior distributions and modal estimates in Markov mixture models , 1996 .

[7]  Ronald Rosenfeld,et al.  Adaptive Statistical Language Modeling; A Maximum Entropy Approach , 1994 .

[8]  A. B. Poritz,et al.  Linear predictive hidden Markov models and the speech signal , 1982, ICASSP.

[9]  Marina Meila,et al.  An Accelerated Chow and Liu Algorithm: Fitting Tree Distributions to High-Dimensional Sparse Data , 1999, ICML.

[10]  Daniel S. Wilks,et al.  Realizations of Daily Weather in Forecast Seasonal Climate , 2002 .

[11]  J. Ashford,et al.  Multi-variate probit analysis. , 1970, Biometrics.

[12]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  José Manuel Gutiérrez,et al.  Bayesian Networks for Probabilistic Weather Prediction , 2002, ECAI.

[14]  Michael I. Jordan,et al.  Probabilistic Independence Networks for Hidden Markov Probability Models , 1997, Neural Computation.

[15]  D. Wilks Multisite generalization of a daily stochastic precipitation generation model , 1998 .

[16]  D. Haussler,et al.  Hidden Markov models in computational biology. Applications to protein modeling. , 1993, Journal of molecular biology.

[17]  Andrew W. Moore,et al.  Using Tarjan's Red Rule for Fast Dependency Tree Construction , 2002, NIPS.

[18]  Upmanu Lall,et al.  A k‐nearest‐neighbor simulator for daily precipitation and other weather variables , 1999 .

[19]  Lain L. MacDonald,et al.  Hidden Markov and Other Models for Discrete- valued Time Series , 1997 .

[20]  Yoshua Bengio,et al.  Markovian Models for Sequential Data , 2004 .

[21]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[22]  L. Baum,et al.  Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .

[23]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[24]  Alan Genz,et al.  Numerical computation of rectangular bivariate and trivariate normal and t probabilities , 2004, Stat. Comput..

[25]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[26]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[27]  Peter Guttorp,et al.  A Hidden Markov Model for Space‐Time Precipitation , 1991 .

[28]  David T. Brown,et al.  A Note on Approximations to Discrete Probability Distributions , 1959, Inf. Control..

[29]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[30]  Tom Minka,et al.  A family of algorithms for approximate Bayesian inference , 2001 .

[31]  I. Csiszár $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[32]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[33]  James D. Hamilton Time Series Analysis , 1994 .

[34]  James P. Hughes,et al.  A hidden Markov model for downscaling synoptic atmospheric patterns to precipitation amounts , 2000 .

[35]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[36]  Frank Jensen,et al.  Optimal junction Trees , 1994, UAI.

[37]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[38]  Ross D. Shachter Bayes-Ball: The Rational Pastime (for Determining Irrelevance and Requisite Information in Belief Networks and Influence Diagrams) , 1998, UAI.

[39]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[40]  Yoshua Bengio,et al.  An Input Output HMM Architecture , 1994, NIPS.

[41]  G. McCabe,et al.  Simulation of precipitation by weather type analysis , 1991 .

[42]  P. Guttorp,et al.  A non‐homogeneous hidden Markov model for precipitation occurrence , 1999 .

[43]  David R. Cox,et al.  A simple spatial-temporal model of rainfall , 1988, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[44]  Padhraic Smyth,et al.  Conditional Chow-Liu Tree Structures for Modeling Discrete-Valued Vector Time Series , 2004, UAI.

[45]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[46]  James P. Hughes,et al.  Incorporating Spatial Dependence and Atmospheric Data in a Model of Precipitation , 1994 .

[47]  M. Do Fast approximation of Kullback-Leibler distance for dependence trees and hidden Markov models , 2003, IEEE Signal Processing Letters.

[48]  A. Bárdossy,et al.  SPACE-TIME MODEL FOR DAILY RAINFALL USING ATMOSPHERIC CIRCULATION PATTERNS , 1992 .

[49]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[50]  Nathan Srebro,et al.  Maximum likelihood bounded tree-width Markov networks , 2001, Artif. Intell..

[51]  Adam L. Berger,et al.  A Maximum Entropy Approach to Natural Language Processing , 1996, CL.

[52]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[53]  Frank Bretz,et al.  Comparison of Methods for the Computation of Multivariate t Probabilities , 2002 .

[54]  Nir Friedman,et al.  Learning Belief Networks in the Presence of Missing Values and Hidden Variables , 1997, ICML.

[55]  S. Chib,et al.  Analysis of multivariate probit models , 1998 .

[56]  S. L. Scott Bayesian Methods for Hidden Markov Models , 2002 .

[57]  David Heckerman,et al.  Knowledge Representation and Inference in Similarity Networks and Bayesian Multinets , 1996, Artif. Intell..

[58]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[59]  Emmanuel Lesaffre,et al.  Existence and Uniqueness of the Maximum Likelihood Estimator for a Multivariate Probit Model , 1992 .

[60]  A. Berchtold The double chain markov model , 1999 .

引用
A comparison of three multi-site statistical downscaling models for daily rainfall in the North China Plain
Theoretical and Applied Climatology
2013
Unsupervised learning algorithms for intrusion detection
NOMS 2008 - 2008 IEEE Network Operations and Management Symposium
2008
Stochastic Precipitation Generation for the Potomac River Basin Using Hidden Markov Models
2020
Stock prediction: an event-driven approach based on bursty keywords
Frontiers of Computer Science in China
2009
Statistical downscaling of daily precipitation over Greece
2008
Towards a Time Series Approach for the Classification and Evaluation of Collaborative Activities
Comput. Informatics
2015
Efficient Bayesian estimation of the multivariate Double Chain Markov Model
Stat. Comput.
2013
Modelling runoff with statistically downscaled daily site, gridded and catchment rainfall series
2013
Removing Object Bouncing from Indoor Tracking Data
ACM Southeast Regional Conference
2017
Learning-Based Cleansing for Indoor RFID Data
SIGMOD Conference
2016
Products of Hidden Markov Models: It Takes N>1 to Tango
UAI
2009
Hierarchical Bayesian Approach to a Multi-Site Hidden Markov Model
Commun. Stat. Simul. Comput.
2014
Network anomaly detection by continuous hidden markov models: An evolutionary programming approach
Intell. Data Anal.
2015
Composable, Distributed-state Models for High-dimensional Time Series
2009
Integrating Multiple Data Sources for Stock Prediction
WISE
2008
Flexible and Accurate Detection of Genomic Copy-Number Changes from aCGH
PLoS Comput. Biol.
2007
Bayesian nonhomogeneous Markov models via Pólya-Gamma data augmentation with applications to rainfall modeling
1701.02856
2017
Subseasonal‐to‐interdecadal variability of the Australian monsoon over North Queensland
2006
Cognitive Load Estimation in the Wild
CHI
2018
Multi-period classification: learning sequent classes from temporal domains
Data Mining and Knowledge Discovery
2014