Partially Hidden Markov Chain Linear Autoregressive model: inference and forecasting

Time series subject to change in regime have attracted much interest in domains such as econometry, finance or meteorology. For discrete-valued regimes, some models such as the popular Hidden Markov Chain (HMC) describe time series whose state process is unknown at all time-steps. Sometimes, time series are firstly labelled thanks to some annotation function. Thus, another category of models handles the case with regimes observed at all time-steps. We present a novel model which addresses the intermediate case: (i) state processes associated to such time series are modelled by Partially Hidden Markov Chains (PHMCs); (ii) a linear autoregressive (LAR) model drives the dynamics of the time series, within each regime. We describe a variant of the expection maximization (EM) algorithm devoted to PHMC-LAR model learning. We propose a hidden state inference procedure and a forecasting function that take into account the observed states when existing. We assess inference and prediction performances, and analyze EM convergence times for the new model, using simulated data. We show the benefits of using partially observed states to decrease EM convergence times. A fully labelled scheme with unreliable labels also speeds up EM. This offers promising prospects to enhance PHMCLAR model selection. We also point out the robustness of PHMC-LAR to labelling errors in inference task, when large training datasets and moderate labelling error rates are considered. Finally, we highlight the remarkable robustness to error labelling in the prediction task, over the whole range of error rates.

[1]  Reid G. Simmons,et al.  Unsupervised learning of probabilistic models for robot navigation , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[2]  E. S. Gardner EXPONENTIAL SMOOTHING: THE STATE OF THE ART, PART II , 2006 .

[3]  Mouhcine Rabi,et al.  Recognition of cursive Arabic handwritten text using embedded training based on HMMs , 2016, 2016 International Conference on Engineering & MIS (ICEMIS).

[4]  Stefan Wrobel,et al.  Active Learning of Partially Hidden Markov Models , 2001 .

[5]  David Ubilava,et al.  Forecasting ENSO with a smooth transition autoregressive model , 2013, Environ. Model. Softw..

[6]  Thierry Denoeux,et al.  Making Use of Partial Knowledge About Hidden States in HMMs: An Approach Based on Belief Functions , 2014, IEEE Transactions on Fuzzy Systems.

[7]  P. Phillips Testing for a Unit Root in Time Series Regression , 1988 .

[8]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[9]  Andrew J. Filardo Business-Cycle Phases and Their Transitional Dynamics , 1994 .

[10]  Björn W. Schuller,et al.  Hidden Markov model-based speech emotion recognition , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[11]  Susan M. Bridges,et al.  Incremental learning of discrete hidden markov models , 2005 .

[12]  M. G. Kendall,et al.  A Study in the Analysis of Stationary Time-Series. , 1955 .

[13]  Jens Timmer,et al.  A new approximate likelihood estimator for ARMA-filtered hidden Markov models , 2000, IEEE Trans. Signal Process..

[14]  A Markov Regime-Switching Framework Application for Describing El Niño Southern Oscillation (ENSO) Patterns , 2015 .

[15]  James D. Hamilton A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle , 1989 .

[16]  Jr. G. Forney,et al.  Viterbi Algorithm , 1973, Encyclopedia of Machine Learning.

[17]  J. Ekanayake,et al.  Multistep short-term wind speed prediction using nonlinear auto-regressive neural network with exogenous variable selection , 2020, Alexandria Engineering Journal.

[18]  Rob J Hyndman,et al.  Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation , 2016 .

[19]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[20]  Michael P. Clements,et al.  A Comparison of the Forecast Performance of Markov�?Switching and Threshold Autoregressive Models of Us Gnp , 1998 .

[21]  V. Monbet,et al.  Comparison of hidden and observed regime-switching autoregressive models for (u, v)-components of wind fields in the northeastern Atlantic , 2016 .

[22]  Christoph Richter,et al.  Action Recognition in Assembly for Human-Robot-Cooperation using Hidden Markov Models , 2018 .

[23]  V. Monbet,et al.  Non-homogeneous hidden Markov-switching models for wind time series , 2015 .

[24]  W. Fuller,et al.  Distribution of the Estimators for Autoregressive Time Series with a Unit Root , 1979 .

[25]  Ying Wang,et al.  Application of a New Hybrid Model with Seasonal Auto-Regressive Integrated Moving Average (ARIMA) and Nonlinear Auto-Regressive Neural Network (NARNN) in Forecasting Incidence Cases of HFMD in Shenzhen, China , 2014, PloS one.

[26]  Ruqiang Yan,et al.  Bearing Degradation Evaluation Using Improved Cross Recurrence Quantification Analysis and Nonlinear Auto-Regressive Neural Network , 2019, IEEE Access.

[27]  Everette S. Gardner,et al.  Exponential smoothing: The state of the art , 1985 .

[28]  Yun Fu,et al.  ARMA-HMM: A new approach for early recognition of human activity , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[29]  Chang‐Jin Kim,et al.  Dynamic linear models with Markov-switching , 1994 .

[30]  P. Phillips,et al.  Testing the null hypothesis of stationarity against the alternative of a unit root: How sure are we that economic time series have a unit root? , 1992 .

[31]  Deepti Chopra,et al.  Named Entity Recognition using Hidden Markov Model (HMM) , 2012 .

[32]  James D. Hamilton Analysis of time series subject to changes in regime , 1990 .

[33]  Davood Gharavian,et al.  A novel method for day-ahead solar power prediction based on hidden Markov model and cosine similarity , 2020, Soft Comput..

[34]  M. Kendall,et al.  A Study in the Analysis of Stationary Time-Series. , 1955 .

[35]  A. Degtyarev,et al.  Evaluation of Hydrodynamic Pressures for Autoregressive Model of Irregular Waves , 2019, Contemporary Ideas on Ship Stability.