Strong Separation of Learning Classes

Suppose LC1 and LC2 are two machine learning classes each based on a criterion of success. Suppose, for every machine which learns a class of functions according to the LC1 criterion of success, there is a machine which learns this class according to the LC2 criterion. In the case where the converse does not hold LC1 is said to be separated from LC2. It is shown that for many such separated learning classes from the literature a much stronger separation holds: (∀∁ ∈ LC1)(∃∁′ ∈ (LC2−LC1))[∁′ ⊃ ∁]. It is also shown that there is a pair of separated learning classes from the literature for which the stronger separation just above does not hold. A philosophical heuristic toward the design of artificially intelligent learning programs is presented with each strong separation result.

[1]  Manuel Blum.  On the Size of Machines , 1967, Inf. Control..

[2]  Rolf Wiehagen,et al.  Research in the theory of inductive inference by GDR mathematicians - A survey , 1980, Inf. Sci..

[3]  K. Popper,et al.  The Logic of Scientific Discovery , 1960 .

[4]  Keh-Jiann Chen,et al.  Tradeoffs in machine inductive inference , 1981 .

[5]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[6]  Ya. M. Barzdin,et al.  Towards a Theory of Inductive Inference (in Russian) , 1973, MFCS.

[7]  Hartley Rogers,et al.  Gödel numberings of partial recursive functions , 1958, Journal of Symbolic Logic.

[8]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[9]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[10]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[11]  John Case.  The power of vacillation , 1988, COLT '88.

[12]  James S. Royer.  A Connotational Theory of Program Structure , 1987, Lecture Notes in Computer Science.

[13]  John Case,et al.  Complexity Issues for Vacillatory Function Identification , 1991, FSTTCS.

[14]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[15]  Gregory A. Riccardi.  The Independence of Control Structures in Abstract Programming Systems , 1981, J. Comput. Syst. Sci..

[16]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[17]  Mark A. Fulk.  A study of inductive inference machines , 1986 .

[18]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[19]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[20]  Yves Marcoux.  Composition is almost as good as s-1-1 , 1989, [1989] Proceedings. Structure in Complexity Theory Fourth Annual Conference.

[21]  Z. Kopal,et al.  Numerical analysis , 1955 .

[22]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[23]  Jerome A. Feldman,et al.  Some Decidability Results on Grammatical Inference and Complexity , 1972, Inf. Control..