Forward estimation for ergodic time series

The forward estimation problem for stationary and ergodic time series {Xn}n=0∞ taking values from a finite alphabet X is to estimate the probability that Xn+1=x based on the observations Xi, 0⩽i⩽n without prior knowledge of the distribution of the process {Xn}. We present a simple procedure gn which is evaluated on the data segment (X0,…,Xn) and for which, error(n)=|gn(x)−P(Xn+1=x|X0,…,Xn)|→0 almost surely for a subclass of all stationary and ergodic time series, while for the full class the Cesaro average of the error tends to zero almost surely and moreover, the error tends to zero in probability.

[1]  D. A. Sprott,et al.  Foundations of Statistical Inference. , 1972 .

[2]  Robert M. Gray,et al.  Probability, Random Processes, And Ergodic Properties , 1987 .

[3]  S. Kalikow,et al.  Random markov processes and uniform martingales , 1990 .

[4]  D. Ornstein.  Guessing the next output of a stationary process , 1978 .

[5]  Benjamin Weiss,et al.  Forecasting for Stationary Binary Time Series , 2003, ArXiv.

[6]  D. Ornstein.  Ergodic theory, randomness, and dynamical systems , 1974 .

[7]  G. Lugosi,et al.  Strategies for Sequential Prediction of Stationary Time Series , 2000 .

[8]  Paul H. Algoet,et al.  The strong law of large numbers for sequential decisions under uncertainty , 1994, IEEE Trans. Inf. Theory.

[9]  Paul C. Shields,et al.  Cutting and stacking: A method for constructing stationary processes , 1991, IEEE Trans. Inf. Theory.

[10]  Sidney J. Yakowitz,et al.  Weakly convergent nonparametric forecasting of stationary time series , 1997, IEEE Trans. Inf. Theory.

[11]  Ferenc Szidarovszky,et al.  Modeling uncertainty : an examination of stochastic theory, methods, and applications , 2002 .

[12]  László Györfi,et al.  A Simple Randomized Algorithm for Consistent Sequential Prediction of Ergodic time Series , 1998 .

[13]  B. Weiss.  Single Orbit Dynamics , 1999 .

[14]  Gusztáv Morvai,et al.  Guessing the output of a stationary binary time series , 2007, ArXiv.

[15]  László Györfi,et al.  A simple randomized algorithm for sequential prediction of ergodic time series , 1999, IEEE Trans. Inf. Theory.

[16]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[17]  Kazuoki Azuma.  WEIGHTED SUMS OF CERTAIN DEPENDENT RANDOM VARIABLES , 1967 .

[18]  László Györfi,et al.  Nonparametric inference for ergodic, stationary time series , 1996, ArXiv.

[19]  Philip T. Maker.  The ergodic theorem for a sequence of functions , 1940 .

[20]  László Györfi,et al.  Limits to Consistent On-Line Forecasting for Ergodic Time Series , 1998, IEEE Trans. Inf. Theory.

[21]  Paul H. Algoet,et al.  Universal Schemes for Learning the Best Nonlinear Predictor Given the Infinite Past and Side Information , 1999, IEEE Trans. Inf. Theory.

[22]  P. Algoet.  UNIVERSAL SCHEMES FOR PREDICTION, GAMBLING AND PORTFOLIO SELECTION' , 1992 .