Exact and approximate hidden Markov chain filters based on discrete observations

Abstract We consider a Hidden Markov Model (HMM) where the integrated continuous-time Markov chain can be observed at discrete time points perturbed by a Brownian motion. The aim is to derive a filter for the underlying continuous-time Markov chain. The recursion formula for the discrete-time filter is easy to derive, however involves densities which are very hard to obtain. In this paper we derive exact formulas for the necessary densities in the case the state space of the HMM consists of two elements only. This is done by relating the underlying integrated continuous-time Markov chain to the so-called asymmetric telegraph process and by using recent results on this process. In case the state space consists of more than two elements we present three different ways to approximate the densities for the filter. The first approach is based on the continuous filter problem. The second approach is to derive a PDE for the densities and solve it numerically. The third approach is a crude discrete time approximation of the Markov chain. All three approaches are compared in a numerical study.

[1]  On the Asymmetric Telegraph Processes , 2014, Journal of Applied Probability.

[2]  Nicole Bäuerle,et al.  The Relaxed Investor with Partial Information , 2012, SIAM J. Financial Math..

[3]  R. Frey,et al.  PORTFOLIO OPTIMIZATION UNDER PARTIAL INFORMATION WITH EXPERT OPINIONS , 2012 .

[4]  U. Rieder,et al.  Markov Decision Processes with Applications to Finance , 2011 .

[5]  Ralf Wunderlich,et al.  Optimal portfolio policies under bounded expected loss and partial information , 2010, Math. Methods Oper. Res..

[6]  Tak Kuen Siu,et al.  Portfolio Selection in the Enlarged Markovian Regime-Switching Market , 2009, SIAM J. Control. Optim..

[7]  S. Frühwirth-Schnatter,et al.  Markov Chain Monte Carlo Methods for Parameter Estimation in Multidimensional Continuous Time Markov Switching Models , 2010 .

[8]  Quasi-Exact Approximation of Hidden Markov Chain Filters , 2009 .

[9]  Markus Hahn,et al.  Parameter estimation in continuous time Markov switching models: a semi-continuous Markov chain Monte Carlo approach , 2009 .

[10]  Hajo Holzmann,et al.  Testing for two states in a hidden Markov model , 2008 .

[11]  Oswaldo Luiz V. Costa,et al.  A generalized multi-period mean-variance portfolio optimization with Markov switching parameters , 2008, Autom..

[12]  Robert J. Elliott,et al.  Moment Based Regression Algorithms for Drift and Volatility Estimation in Continuous-Time Markov Switching Models , 2008 .

[13]  Erhan Bayraktar,et al.  Sequential tracking of a hidden Markov chain using point process observations , 2007 .

[14]  B. Fristedt,et al.  Filtering and Prediction: A Primer , 2007 .

[15]  N. Yoshida,et al.  Estimation for the discretely observed telegraph process , 2006, math/0612784.

[16]  U. Rieder,et al.  Portfolio optimization with unobservable Markov-modulated drift process , 2005, Journal of Applied Probability.

[17]  Ulrich G. Haussmann,et al.  Optimizing the terminal wealth under partial information: The drift process as a continuous time Markov chain , 2004, Finance Stochastics.

[18]  B. Sericola,et al.  A second-order Markov-modulated fluid queue with linear service rate , 2004, Journal of Applied Probability.

[19]  Nicole Bäuerle,et al.  Portfolio optimization with Markov-modulated stock prices and interest rates , 2004, IEEE Transactions on Automatic Control.

[20]  Gang George Yin,et al.  Markowitz's mean-variance portfolio selection with regime switching: from discrete-time models to their continuous-time limits , 2004, IEEE Transactions on Automatic Control.

[21]  E. Otranto,et al.  A NONPARAMETRIC BAYESIAN APPROACH TO DETECT THE NUMBER OF REGIMES IN MARKOV SWITCHING MODELS , 2002 .

[22]  S. Frühwirth-Schnatter.  Fully Bayesian Analysis of Switching Gaussian State Space Models , 2001 .

[23]  S. Iacus.  Statistic Analysis of the Inhomogeneous Telegrapher's Process , 2000, math/0011059.

[24]  Vidyadhar G. Kulkarni,et al.  Fluid models for single buffer systems , 1998 .

[25]  T. Rydén,et al.  Stylized Facts of Daily Return Series and the Hidden Markov Model , 1998 .

[26]  S. Eddy.  Hidden Markov models. , 1996, Current opinion in structural biology.

[27]  T. Rydén.  An EM algorithm for estimation in Markov-modulated Poisson processes , 1996 .

[28]  John B. Moore,et al.  Hidden Markov Models: Estimation and Control , 1994 .

[29]  Y. Kabanov,et al.  Mean-square hedging of options on a stock with Markov volatilities , 1994 .

[30]  Peter E. Kloeden,et al.  Higher Order Approximate Markov Chain Filters , 1993 .

[31]  Thomas E. Stern,et al.  Analysis of separable Markov-modulated rate models for information-handling systems , 1991, Advances in Applied Probability.

[32]  Martin Berzins,et al.  A Method for the Spatial Discretization of Parabolic Equations in One Space Variable , 1990, SIAM J. Sci. Comput..

[33]  David M. Lucantoni,et al.  A Markov Modulated Characterization of Packetized Voice and Data Traffic and Related Statistical Multiplexer Performance , 1986, IEEE J. Sel. Areas Commun..

[34]  Y. C. Yao,et al.  Estimation of noisy telegraph processes: Nonlinear filtering versus nonlinear smoothing , 1985, IEEE Trans. Inf. Theory.

[35]  M. Kac.  A stochastic model related to the telegrapher's equation , 1974 .