Model Theory and Machine Learning

About 25 years ago, it came to light that a single combinatorial property determines both an important dividing line in model theory (NIP) and machine learning (PAC-learnability). The following years saw a fruitful exchange of ideas between PAC learning and the model theory of NIP structures. In this article, we point out a new and similar connection between model theory and machine learning, this time developing a correspondence between \emph{stability} and learnability in various settings of \emph{online learning.} In particular, this gives many new examples of mathematically interesting classes which are learnable in the online setting.

[1]  N. Littlestone Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[2]  E. Bouscaren Model theory and algebraic geometry : an introduction to E. Hrushovski's proof of the geometric Mordell-Lang conjecture , 1998 .

[3]  A. Chernikov,et al.  Externally definable sets and dependent pairs , 2010, Israel Journal of Mathematics.

[4]  Ambuj Tewari,et al.  Online Learning: Beyond Regret , 2010, COLT.

[5]  W. Baur Elimination of quantifiers for modules , 1976 .

[6]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[7]  Siddharth Bhaskar THICKET DENSITY , 2021, The Journal of Symbolic Logic.

[8]  Roi Livni,et al.  Honest Compressions and Their Application to Compression Schemes , 2013, COLT.

[9]  Carol Wood,et al.  Notes on the stability of separably closed fields , 1979, Journal of Symbolic Logic.

[10]  Rahim Moosa Model Theory and Complex Geometry , 2009 .

[11]  田中 克己,et al.  ω-stable groups , 1988 .

[12]  W. Hodges CLASSIFICATION THEORY AND THE NUMBER OF NON‐ISOMORPHIC MODELS , 1980 .

[13]  Michael C. Laskowski,et al.  Vapnik-Chervonenkis classes of definable sets , 1992 .

[14]  Z. Sela Diophantine geometry over groups VIII: Stability , 2006, math/0609096.

[15]  Shai Ben-David,et al.  Agnostic Online Learning , 2009, COLT.

[16]  Michael C. Laskowski,et al.  Compression Schemes, Stable Definable Families, and o-Minimal Structures , 2010, Discret. Comput. Geom..