Shrinkage Estimation for Multivariate Hidden Markov Models

ABSTRACT Motivated from a changing market environment over time, we consider high-dimensional data such as financial returns, generated by a hidden Markov model that allows for switching between different regimes or states. To get more stable estimates of the covariance matrices of the different states, potentially driven by a number of observations that are small compared to the dimension, we modify the expectation–maximization (EM) algorithm so that it yields the shrinkage estimators for the covariance matrices. The final algorithm turns out to reproduce better estimates not only for the covariance matrices but also for the transition matrix. It results into a more stable and reliable filter that allows for reconstructing the values of the hidden Markov chain. In addition to a simulation study performed in this article, we also present a series of theoretical results that include dimensionality asymptotics and provide the motivation for certain techniques used in the algorithm. Supplementary materials for this article are available online.

[1]  Joseph Tadjuidje Kamgaing,et al.  On geometric ergodicity of CHARME models , 2010 .

[2]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[3]  T. Rydén,et al.  Stylized Facts of Daily Return Series and the Hidden Markov Model , 1998 .

[4]  H. Ombao,et al.  The generalized shrinkage estimator for the analysis of functional connectivity of brain signals , 2011, 1108.3187.

[5]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[6]  J. Zakoian,et al.  Stationarity of Multivariate Markov-Switching ARMA Models , 2001 .

[7]  P. Doukhan,et al.  A new weak dependence condition and applications to moment inequalities , 1999 .

[8]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.

[9]  Kevin P. Murphy,et al.  Modeling changing dependency structure in multivariate time series , 2007, ICML '07.

[10]  C. Francq,et al.  On White Noises Driven by Hidden Markov Chains , 1997 .

[11]  J. Franke,et al.  Mixtures of nonparametric autoregressions , 2011 .

[12]  Alessio Sancetta.  Sample covariance shrinkage for high dimensional dependent data , 2006 .

[13]  Joseph Tadjuidje Kamgaing.  Maximum Likelihood Estimators for Multivariate Hidden Markov Mixture Models , 2013 .

[14]  P. Green.  On Use of the EM Algorithm for Penalized Likelihood Estimation , 1990 .

[15]  Jürgen Franke,et al.  Fitting autoregressive models to EEG time series: An empirical comparison of estimates of the order , 1985, IEEE Trans. Acoust. Speech Signal Process..

[16]  R. Okafor.  Maximum likelihood estimation from incomplete data , 1987 .

[17]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[18]  T. Cai,et al.  A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation , 2011, 1102.2233.

[19]  Chenlei Leng,et al.  Bayesian adaptive Lasso , 2010, Annals of the Institute of Statistical Mathematics.

[20]  Minxian Yang.  SOME PROPERTIES OF VECTOR AUTOREGRESSIVE PROCESSES WITH MARKOV-SWITCHING COEFFICIENTS , 2000, Econometric Theory.

[21]  Hernando Ombao,et al.  Functional connectivity: Shrinkage estimation and randomization test , 2010, NeuroImage.

[22]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[23]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[24]  Jianhua Z. Huang,et al.  Regularized parameter estimation of high dimensional t distribution , 2009 .

[25]  N. Hengartner,et al.  Structural learning with time‐varying components: tracking the cross‐section of financial time series , 2005 .

[26]  Eric Moulines,et al.  Inference in hidden Markov models , 2010, Springer series in statistics.

[27]  Jianqing Fan,et al.  NETWORK EXPLORATION VIA THE ADAPTIVE LASSO AND SCAD PENALTIES. , 2009, The annals of applied statistics.

[28]  Rainer von Sachs,et al.  Shrinkage estimation in the frequency domain of multivariate time series , 2009, J. Multivar. Anal..

[29]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[30]  R. Jagannathan,et al.  Generalized Methods of Moments , 2002 .

[31]  Olivier Ledoit,et al.  Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices , 2011, 1207.5322.

[32]  R. Douc,et al.  Asymptotic properties of the maximum likelihood estimator in autoregressive models with Markov regime , 2004, math/0503681.

[33]  R. C. Bradley.  Basic properties of strong mixing conditions. A survey and some open questions , 2005, math/0511078.

[34]  D. Andrews.  Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation , 1991 .

[35]  W. Härdle,et al.  Applied Multivariate Statistical Analysis , 2003 .

[36]  J. Ibrahim,et al.  Genomewide Multiple-Loci Mapping in Experimental Crosses by Iterative Adaptive Penalized Regression , 2010, Genetics.

[37]  Ravi Jagannathan,et al.  Generalized Method of Moments: Applications in Finance , 2002 .