Predictive Learning Models for Concept Drift

Concept drift means that the concept about which data is obtained may shift from time to time, each time after some minimum permanence. Except for this minimum permanence, the concept shifts may not have to satisfy any further requirements and may occur infinitely often. Within this work is studied to what extent it is still possible to predict or learn values for a data sequence produced by drifting concepts. Various ways to measure the quality of such predictions, including martingale betting strategies and density and frequency of correctness, are introduced and compared with one another. For each of these measures of prediction quality, for some interesting concrete classes, (nearly) optimal bounds on permanence for attaining learnability are established. The concrete classes, from which the drifting concepts are selected, include regular languages accepted by finite automata of bounded size, polynomials of bounded degree, and sequences defined by recurrence relations of bounded size. Some important, restricted cases of drifts are also studied, for example, the case where the intervals of permanence are computable. In the case where the concepts shift only

[1]  Setsuo Arikawa,et al.  Pattern Inference , 1995, GOSLER Final Report.

[2]  D. C. Cooper,et al.  Theory of Recursive Functions and Effective Computability , 1969, The Mathematical Gazette.

[3]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[4]  Tunc Geveci,et al.  Advanced Calculus , 2014, Nature.

[5]  James S. Royer,et al.  Inductive Inference of Approximations , 1986, Inf. Control..

[6]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[7]  Thomas Zeugmann,et al.  One-Sided Error Probabilistic Inductive Inference and Reliable Frequency Identification , 1991, Inf. Comput..

[8]  Sanjay Jain,et al.  Approximate Inference and Scientific Method , 1990, ALT.

[9]  C. Schnorr.  Zufälligkeit und Wahrscheinlichkeit , 1971 .

[10]  Ashwin Ram,et al.  Dynamically Adjusting Concepts to Accommodate Changing Contexts , 1994, AAAI 1994.

[11]  Steffen Lange,et al.  Algorithmic Learning for Knowledge-Based Systems: Gosler Final Report , 1995 .

[12]  Shai Ben-David,et al.  Learning Changing Concepts by Exploiting the Structure of Change , 1996, COLT '96.

[13]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[14]  Miroslav Kubat.  A machine learning-based approach to load balancing in computer networks , 1992 .

[15]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[16]  Frank Stephan,et al.  Robust Learning with Infinite Additional Information , 1997, EuroCOLT.

[17]  N. Littlestone.  Mistake bounds and logarithmic linear-threshold learning algorithms , 1990 .

[18]  Yishay Mansour,et al.  Learning Under Persistent Drift , 1997, EuroCOLT.

[19]  Avrim Blum,et al.  Learning switching concepts , 1992, COLT '92.

[20]  S. Wrobel.  Concept Formation and Knowledge Revision , 1994, Springer US.