On Aggregating Teams of Learning Machines

The present paper studies the problem of when a team of learning machines can be aggregated into a single learning machine without any loss in learning power. The main results concern aggregation ratios for vacillatory identification of languages from texts. For a positive integer n, a machine is said to TxtFexn-identify a language L just in case the machine converges to upto n grammars for L on any text for L. For such identification criteria, the aggregation ratio is derived for the n=2 case. It is shown that the collection of languages that can be TxtFex2 identified by teams with success ratio greater than 5/6 are the same as those collections of languages that can be TxtFex2-identified by a single machine. It is also established that 5/6 is indeed the cut-off point by showing that there are collections of languages that can be TxtFex2-identified by a team employing 6 machines, at least 5 of which are required to be successful, but cannnot be TxtFex2-identified by any single machine. Additionally, aggregation ratios are also derived for finite identification of languages from positive data and for numerous criteria involving language learning from both positive and negative data.

[1]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[2]  Arun Sharma,et al.  Probability is more powerful than team for language identification from positive data , 1993, COLT '93.

[3]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[4]  Ya. M. Barzdin,et al.  Towards a Theory of Inductive Inference (in Russian) , 1973, MFCS.

[5]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[6]  Carl H. Smith,et al.  The Power of Pluralism for Automatic Program Synthesis , 1982, JACM.

[7]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[8]  Hartley Rogers,et al.  Gödel numberings of partial recursive functions , 1958, Journal of Symbolic Logic.

[9]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[10]  R. V. Freivald.  Functions Computable in the Limit by Probabilistic Machines , 1974, MFCS.

[11]  Arun Sharma,et al.  Finite learning by a “team” , 1990, COLT '90.

[12]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[13]  Efim B. Kinber,et al.  On a Theory of Inductive Inference , 1977, FCT.

[14]  Carl H. Smith,et al.  Probability and Plurality for Aggregations of Learning Machines , 1987, Inf. Comput..

[15]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[16]  Daniel N. Osherson,et al.  Aggregating Inductive Expertise , 1986, Inf. Control..

[17]  Leonard Pitt,et al.  Relations between probabilistic and team one-shot learners (extended abstract) , 1991, COLT '91.

[18]  Mahendran Velauthapillai.  Inductive inference with bounded number of mind changes , 1989, COLT '89.

[19]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[20]  John Case.  The power of vacillation , 1988, COLT '88.

[21]  Robert P. Daley.  Inductive inference hierarchies: probabilistic vs pluralistic strategies , 1985, Mathematical Methods of Specification and Synthesis of Software Systems.

[22]  Bala Kalyanasundaram,et al.  Breaking the probability ½ barrier in FIN-type learning , 1992, COLT '92.

[23]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[24]  Arun Sharma,et al.  Computational Limits on Team Identification of Languages , 1996, Inf. Comput..

[25]  Bala Kalyanasundaram,et al.  Breaking the Probability 1/2 Barrier in FIN-Type Learning , 1995, J. Comput. Syst. Sci..

[26]  Leonard Pitt,et al.  A Characterization of Probabilistic Inference , 1984, FOCS.