Relevant states and memory in Markov chain bootstrapping and simulation

Markov chain theory is proving to be a powerful approach to bootstrap highly nonlinear time series. In this work we provide a method to estimate the memory of a Markov chain (i.e. its order) and to identify its relevant states. In particular the choice of memory lags and the aggregation of irrelevant states are obtained by looking for regularities in the transition probabilities. Our approach is based on an optimization model. More specifically we consider two competing objectives that a researcher will in general pursue when dealing with bootstrapping: preserving the “structural” similarity between the original and the simulated series and assuring a controlled diversification of the latter. A discussion based on information theory is developed to define the desirable properties for such optimal criteria. Two numerical tests are developed to verify the effectiveness of the method proposed here.

[1]  Joseph P. Romano,et al.  The stationary bootstrap , 1994 .

[2]  Meir Feder,et al.  A universal finite memory source , 1995, IEEE Trans. Inf. Theory.

[3]  Soumendra N. Lahiri,et al.  A nonparametric plug-in rule for selecting optimal block lengths for block bootstrap methods ☆ , 2007 .

[4]  H. White,et al.  Data‐Snooping, Technical Trading Rule Performance, and the Bootstrap , 1999 .

[5]  Efstathios Paparoditis,et al.  The Tapered Block Bootstrap for General Statistics from Stationary Sequences , 2002 .

[6]  M. B. Rajarshi.  Bootstrap in Markov-sequences based on estimates of transition density , 1990 .

[7]  F. Dufour,et al.  Approximation of Markov decision processes with general state space , 2012 .

[8]  Yuval Peres,et al.  Two new Markov order estimators , 2005 .

[9]  Abraham Lempel,et al.  A sequential algorithm for the universal coding of finite memory sources , 1992, IEEE Trans. Inf. Theory.

[10]  Countable state Markov process bootstrap , 1999 .

[11]  Andrey L. Vasnev,et al.  Markov chain approximation in bootstrapping autoregressions , 2002 .

[12]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[13]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[14]  Joseph P. Romano,et al.  A General Resampling Scheme for Triangular Arrays of $\alpha$-Mixing Random Variables with Application to the Problem of Spectral Density Estimation , 1992 .

[15]  I. Csiszár,et al.  The consistency of the BIC Markov order estimator , 2000 .

[16]  Efstathios Paparoditis,et al.  Tapered block bootstrap , 2001 .

[17]  H. White,et al.  Automatic Block-Length Selection for the Dependent Bootstrap , 2004 .

[18]  Neri Merhav,et al.  On the estimation of the order of a Markov chain and universal data compression , 1989, IEEE Trans. Inf. Theory.

[19]  Carsten Hartmann,et al.  Markov State Models for Rare Events in Molecular Dynamics , 2013, Entropy.

[20]  JORMA RISSANEN,et al.  A universal data compression system , 1983, IEEE Trans. Inf. Theory.

[21]  Robert Tibshirani,et al.  An Introduction to the Bootstrap CHAPMAN & HALL/CRC , 1993 .

[22]  E. Carlstein.  The Use of Subseries Values for Estimating the Variance of a General Statistic from a Stationary Sequence , 1986 .

[23]  P. Hall,et al.  On blocking rules for the bootstrap with dependent data , 1995 .

[24]  Michael K. Ng,et al.  Higher-order multivariate Markov chains and their applications , 2008 .

[25]  John C. Kieffer,et al.  Strongly consistent code-based identification and order estimation for constrained finite-state model classes , 1993, IEEE Trans. Inf. Theory.

[26]  Robert Tibshirani,et al.  Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy , 1986 .

[27]  Aurélien Garivier,et al.  A minimum description length approach to hidden Markov models with Poisson and Gaussian emissions. Application to order identification , 2009 .

[28]  D. White.  Infinite horizon Markov decision processes with unknown or variable discount factors , 1987 .

[29]  D. Freedman.  On Bootstrapping Two-Stage Least-Squares Estimates in Stationary Linear Models , 1984 .

[30]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[31]  Vijay S. Pande,et al.  Everything you wanted to know about Markov State Models but were afraid to ask. , 2010, Methods.

[32]  Robert Givan,et al.  Equivalence notions and model minimization in Markov decision processes , 2003, Artif. Intell..

[33]  P. Bühlmann,et al.  Block length selection in the bootstrap for time series , 1999 .

[34]  Alan David Hutson,et al.  Resampling Methods for Dependent Data , 2004, Technometrics.

[35]  Prakash Narayan,et al.  Order estimation and sequential universal data compression of a hidden Markov source by the method of mixtures , 1994, IEEE Trans. Inf. Theory.

[36]  Glen G. Langdon,et al.  Universal modeling and coding , 1981, IEEE Trans. Inf. Theory.

[37]  P. Bühlmann,et al.  Variable Length Markov Chains: Methodology, Computing, and Software , 2004 .

[38]  B. LeBaron,et al.  Simple Technical Trading Rules and the Stochastic Properties of Stock Returns , 1992 .

[39]  Scott Sanner,et al.  Real-time dynamic programming for Markov decision processes with imprecise probabilities , 2016, Artif. Intell..

[40]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[41]  A. Ullah.  Entropy, divergence and distance measures with econometric applications , 1996 .

[42]  Regina Y. Liu.  Moving blocks jackknife and bootstrap capture weak dependence , 1992 .

[43]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[44]  H. Künsch.  The Jackknife and the Bootstrap for General Stationary Observations , 1989 .

[45]  Pravin Varaiya,et al.  Simulation-based optimization of Markov decision processes: An empirical process theory approach , 2010, Autom..

[46]  Sung-Hyuk Cha.  Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions , 2007 .

[47]  Takayoshi Tamura,et al.  New approximate dynamic programming algorithms for large-scale undiscounted Markov decision processes and their application to optimize a production and distribution system , 2016, Eur. J. Oper. Res..

[48]  Jorma Rissanen,et al.  Complexity of strings in the class of Markov sources , 1986, IEEE Trans. Inf. Theory.

[49]  Peter Bühlmann,et al.  Sieve Bootstrap With Variable-Length Markov Chains for Stationary Categorical Time Series , 2002 .

[50]  Gary H. Chao.  Production and availability policies through the Markov Decision Process and myopic methods for contractual and selective orders , 2013, Eur. J. Oper. Res..

[51]  Stanislav Anatolyev.  Electoral behavior of US counties: a panel data approach , 2002 .

[52]  J. Bernardo,et al.  Bayesian Hypothesis Testing: a Reference Approach , 2002 .

[53]  Efstathios Paparoditis,et al.  A MARKOVIAN LOCAL RESAMPLING SCHEME FOR NONPARAMETRIC ESTIMATORS IN TIME SERIES ANALYSIS , 2001, Econometric Theory.

[54]  Benjamin Weiss,et al.  Order estimation of Markov chains , 2005, IEEE Transactions on Information Theory.

[55]  Neri Merhav,et al.  Estimating the number of states of a finite-state source , 1992, IEEE Trans. Inf. Theory.

[56]  Prakasa Rao,et al.  BOOTSTRAPPING A FINITE STATE MARKOV CHAIN , 2016 .

[57]  Roy Cerqueti,et al.  A Tabu Search heuristic procedure in Markov chain bootstrapping , 2013, Eur. J. Oper. Res..

[58]  Jorma Rissanen,et al.  The Minimum Description Length Principle in Coding and Modeling , 1998, IEEE Trans. Inf. Theory.

[59]  D. Politis,et al.  The local bootstrap for Markov processes , 2002 .

[60]  W. McCormick,et al.  Regeneration-based bootstrap for Markov chains , 1993 .

[61]  P. Bühlmann.  Sieve bootstrap for time series , 1997 .

[62]  Neri Merhav,et al.  Universal prediction of individual sequences , 1992, IEEE Trans. Inf. Theory.

[63]  Hyeong Soo Chang,et al.  On functional equations for Kth best policies in Markov decision processes , 2013, Autom..

[64]  I. Basawa,et al.  Asymptotic bootstrap validity for finite markov chains , 1990 .

[65]  P. Bertail,et al.  Second-order properties of regeneration-based bootstrap for atomic Markov chains , 2007 .

[66]  K. Athreya,et al.  Bootstrapping Markov chains: countable case , 1992 .

[67]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[68]  D. Freedman,et al.  Bootstrapping a Regression Equation: Some Empirical Results , 1984 .

[69]  Imre Csiszár.  Large-scale typicality of Markov sample paths and consistency of MDL Order estimators , 2002, IEEE Trans. Inf. Theory.

[70]  Joel L. Horowitz,et al.  Bootstrap Methods for Markov Processes , 2003 .

[71]  Dimitrios G. Pandelis.  Markov decision processes with multidimensional action spaces , 2010, Eur. J. Oper. Res..

[72]  P. Bertail,et al.  Regenerative block bootstrap for Markov chains , 2006 .

[73]  Peter Hall.  Resampling a coverage pattern , 1985 .