Sequential Prediction of Unbounded Stationary Time Series

A simple on-line procedure is considered for the prediction of a real-valued sequence. The algorithm is based on a combination of several simple predictors. If the sequence is a realization of an unbounded stationary and ergodic random process then the average of squared errors converges, almost surely, to that of the optimum, given by the Bayes predictor. An analog result is offered for the classification of binary processes

[1]  M. Loève.  On Almost Sure Convergence , 1951 .

[2]  L. Breiman.  The Individual Ergodic Theorem of Information Theory , 1957 .

[3]  Y. Chow.  Local Convergence of Martingales and the Law of Large Numbers , 1965 .

[4]  D. Ornstein.  Guessing the next output of a stationary process , 1978 .

[5]  P. Algoet.  UNIVERSAL SCHEMES FOR PREDICTION, GAMBLING AND PORTFOLIO SELECTION' , 1992 .

[6]  Neri Merhav,et al.  Universal prediction of individual sequences , 1992, IEEE Trans. Inf. Theory.

[7]  Paul H. Algoet,et al.  The strong law of large numbers for sequential decisions under uncertainty , 1994, IEEE Trans. Inf. Theory.

[8]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[9]  Vladimir Vovk,et al.  A game of prediction with expert advice , 1995, COLT '95.

[10]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[11]  Sidney J. Yakowitz,et al.  Weakly convergent nonparametric forecasting of stationary time series , 1997, IEEE Trans. Inf. Theory.

[12]  Neri Merhav,et al.  Universal Prediction , 1998, IEEE Trans. Inf. Theory.

[13]  László Györfi,et al.  A Simple Randomized Algorithm for Consistent Sequential Prediction of Ergodic time Series , 1998 .

[14]  Dharmendra S. Modha,et al.  Memory-Universal Prediction of Stationary Random Processes , 1998, IEEE Trans. Inf. Theory.

[15]  László Györfi,et al.  A simple randomized algorithm for sequential prediction of ergodic time series , 1999, IEEE Trans. Inf. Theory.

[16]  Andrew C. Singer,et al.  Universal linear prediction by model order weighting , 1999, IEEE Trans. Signal Process..

[17]  G. Lugosi,et al.  Strategies for Sequential Prediction of Stationary Time Series , 2000 .

[18]  M. Feder,et al.  Universal linear least-squares prediction , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[19]  Yuhong Yang.  Combining Different Procedures for Adaptive Regression , 2000, Journal of Multivariate Analysis.

[20]  Claudio Gentile,et al.  Adaptive and Self-Confident On-Line Learning Algorithms , 2000, J. Comput. Syst. Sci..

[21]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[22]  Andrew B. Nobel,et al.  On optimal sequential prediction for general processes , 2003, IEEE Trans. Inf. Theory.

[23]  Tsachy Weissman,et al.  Universal prediction of random binary sequences in a noisy environment , 2004 .

[24]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[25]  László Györfi,et al.  Nonparametric inference for ergodic, stationary time series , 1996, ArXiv.