Online Learning of Non-stationary Sequences

We consider an online learning scenario in which the learner can make predictions on the basis of a fixed set of experts. We derive upper and lower relative loss bounds for a class of universal learning algorithms involving a switching dynamics over the choice of the experts. On the basis of the performance bounds we provide the optimal a priori discretization for learning the parameter that governs the switching dynamics. We demonstrate the new algorithm in the context of wireless networks.

[1]  Yoram Singer,et al.  On‐Line Portfolio Selection Using Multiplicative Updates , 1998, ICML.

[2]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[3]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[4]  Nicolò Cesa-Bianchi,et al.  Gambling in a rigged casino: The adversarial multi-armed bandit problem , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[5]  David Haussler,et al.  Sequential Prediction of Individual Sequences Under General Loss Functions , 1998, IEEE Trans. Inf. Theory.

[6]  J. Rissanen.  Stochastic Complexity and Modeling , 1986 .

[7]  Neri Merhav,et al.  Universal prediction of individual sequences , 1992, IEEE Trans. Inf. Theory.

[8]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[9]  Hari Balakrishnan,et al.  Minimizing Energy for Wireless Web Access with Bounded Slowdown , 2002, MobiCom '02.

[10]  Adam Tauman Kalai,et al.  Finely-competitive paging , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[11]  Vladimir Vovk,et al.  Derandomizing Stochastic Prediction Strategies , 1997, COLT '97.

[12]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[13]  Neri Merhav,et al.  Hierarchical universal coding , 1996, IEEE Trans. Inf. Theory.

[14]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[15]  Mark Herbster,et al.  Tracking the Best Expert , 1995, Machine-mediated learning.

[16]  Parag A. Pathak,et al.  Massachusetts Institute of Technology , 1964, Nature.

[17]  Raphail E. Krichevsky,et al.  The performance of universal encoding , 1981, IEEE Trans. Inf. Theory.

[18]  Carl W. Steinbach.  A Reinforcement-Learning Approach to Power Management , 2002 .

[19]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[20]  Dean P. Foster,et al.  Regret in the On-Line Decision Problem , 1999 .

[21]  L. Goddard.  Information Theory , 1962, Nature.

[22]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[23]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[24]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[25]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[26]  J. A. Salvato.  John wiley & sons. , 1994, Environmental science & technology.