The Forget-me-not Process

We introduce the Forget-me-not Process, an efficient, non-parametric meta-algorithm for online probabilistic sequence prediction for piecewise stationary, repeating sources. Our method works by taking a Bayesian approach to partition a stream of data into postulated task-specific segments, while simultaneously building a model for each task. We provide regret guarantees with respect to piecewise stationary data sources under the logarithmic loss, and validate the method empirically across a range of sequence prediction and task identification problems.

[1]  Frans M. J. Willems,et al.  The context-tree weighting method: basic properties , 1995, IEEE Trans. Inf. Theory.

[2]  Tor Lattimore,et al.  Concentration and Confidence for Discrete Bayesian Sequence Predictors , 2013, ALT.

[3]  E. Koechlin,et al.  Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making , 2012, PLoS biology.

[4]  Jeffrey Scott Vitter,et al.  Random sampling with a reservoir , 1985, TOMS.

[5]  F. Willems,et al.  Live-and-die coding for binary piecewise i.i.d. sources , 1997, Proceedings of IEEE International Symposium on Information Theory.

[6]  Frans M. J. Willems Coding for a binary independent piecewise-identically-distributed source , 1996, IEEE Trans. Inf. Theory.

[7]  Seshadhri Comandur,et al.  Efficient learning algorithms for changing environments , 2009, ICML '09.

[8]  Tamás Linder,et al.  Efficient Tracking of Large Classes of Experts , 2012, IEEE Transactions on Information Theory.

[9]  Mitsuo Kawato,et al.  Multiple Model-Based Reinforcement Learning , 2002, Neural Computation.

[10]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents (Extended Abstract) , 2012, IJCAI.

[11]  Marcus Hutter On Universal Prediction and Bayesian Confirmation , 2007, Theor. Comput. Sci..

[12]  Martha White,et al.  Partition Tree Weighting , 2012, 2013 Data Compression Conference.

[13]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[14]  Raphail E. Krichevsky,et al.  The performance of universal encoding , 1981, IEEE Trans. Inf. Theory.

[15]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[16]  Etienne Koechlin,et al.  Foundations of human reasoning in the prefrontal cortex , 2014, Science.

[17]  Hugo Larochelle,et al.  MADE: Masked Autoencoder for Distribution Estimation , 2015, ICML.

[18]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[19]  Anne G E Collins,et al.  Cognitive control over learning: creating, clustering, and generalizing task-set structure. , 2013, Psychological review.

[20]  Ryan P. Adams,et al.  Bayesian Online Changepoint Detection , 2007, 0710.3742.

[21]  Marc G. Bellemare,et al.  Compress and Control , 2015, AAAI.

[22]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..