A theory of the learnable

Humans appear to be able to learn new concepts without needing to be programmed explicitly in any conventional sense. In this paper we regard learning as the phenomenon of knowledge acquisition in the absence of explicit programming. We give a precise methodology for studying this phenomenon from a computational viewpoint. It consists of choosing an appropriate information gathering mechanism, the learning protocol, and exploring the class of concepts that can be learnt using it in a reasonable (polynomial) number of steps. We find that inherent algorithmic complexity appears to set serious limits to the range of concepts that can be so learnt. The methodology and results suggest concrete principles for designing realistic learning systems.

[1]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.

[2]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[3]  J. Spencer.  Probabilistic Methods in Combinatorics , 1974 .

[4]  Leslie G. Valiant,et al.  A complexity theory based on Boolean algebra , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[5]  Avron Barr,et al.  The Handbook of Artificial Intelligence , 1982 .

[6]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[7]  L. Valiant.  Deductive learning , 1984, Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences.

[8]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[9]  Silvio Micali,et al.  How to construct random functions , 1986, JACM.