The IMP game: Learnability, approximability and adversarial learning beyond $$\Sigma^0_1$$

We introduce a problem set-up we call the Iterated Matching Pennies (IMP) game and show that it is a powerful framework for the study of three problems: adversarial learnability, conventional (i.e., non-adversarial) learnability and approximability. Using it, we are able to derive the following theorems. (1) It is possible to learn by example all of Σ1 ∪ Π1 as well as some supersets; (2) in adversarial learning (which we describe as a pursuit-evasion game), the pursuer has a winning strategy (in other words, Σ1 can be learned adversarially, but Π1 not); (3) some languages in Π 0 1 cannot be approximated by any language in Σ 0 1. We show corresponding results also for Σi and Π 0 i for arbitrary i.

[1]  Ray J. Solomonoff,et al.  Algorithmic Probability|Theory and Applications , 2009 .

[2]  José Hernández-Orallo,et al.  On More Realistic Environment Distributions for Defining, Evaluating and Developing Intelligence , 2011, AGI.

[3]  D. C. Cooper,et al.  Theory of Recursive Functions and Effective Computability , 1969, The Mathematical Gazette.

[4]  Ray J. Solomonoff,et al.  Algorithmic Probability, Heuristic Programming and AGI , 2010, AGI 2010.

[5]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[6]  J. Doug Tygar,et al.  Adversarial machine learning , 2019, AISec '11.

[7]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[8]  David L. Dowe,et al.  Minimum Message Length and Statistically Consistent Invariant (Objective?) Bayesian Probabilistic Inference—From (Medical) “Evidence” , 2008 .

[9]  Sanjay Chawla,et al.  A Game Theoretical Model for Adversarial Learning , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[10]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[11]  David L. Dowe,et al.  Introduction to Ray Solomonoff 85th Memorial Conference , 2011, Algorithmic Probability and Friends.

[12]  John McCarthy,et al.  Algorithmic Probability — Its Discovery — Its Properties and Application to Strong AI , 2009 .

[13]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[14]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[15]  José Hernández-Orallo,et al.  Compression and Intelligence: Social Environments and Communication , 2011, AGI.

[16]  R. Buck,et al.  MICHAEL SCRIVEN AN ESSENTIAL UNPREDICTABILITY IN HUMAN BEHAVIOR * , 2006 .

[17]  David Lewis,et al.  Scriven on human unpredictability , 1966 .

[18]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[19]  Gary William Flake,et al.  The Computational Beauty of Nature: Computer Explorations of Fractals, Chaos, Complex Systems and Adaptation , 1998 .

[20]  Christopher Meek,et al.  Adversarial learning , 2005, KDD '05.

[21]  David L. Dowe,et al.  MML, hybrid Bayesian network graphical models, statistical consistency, invarianc , 2010 .