Probabilistic inductive inference

Inductive inference machines construct programs for total recursive functions given only example values of the functions. <italic>Probabilistic</italic> inductive inference machines are defined, and for various criteria of successful inference, it is asked whether a probabilistic inductive inference machine can infer larger classes of functions if the inference criterion is relaxed to allow inference with probability at least <italic>p</italic>, (0 < <italic>p</italic> < 1) as opposed to requiring certainty. For the most basic criteria of success (<italic>EX</italic> and <italic>BC</italic>), it is shown that any class of functions that can be inferred from examples with probability exceeding 1/2 can be inferred deterministically, and that for probabilities <italic>p</italic> ≤ 1/2 there is a discrete hierarchy of inferability parameterized by <italic>p</italic>. The power of probabilistic inference strategies is characterized by equating the classes of probabilistically inferable functions with those classes that can be inferred by <italic>teams</italic> of inductive inference machines (a parallel model of inference), or by a third model called <italic>frequency</italic> inference.

[1]  Journal of the Association for Computing Machinery , 1961, Nature.

[2]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[3]  James Jay Horning,et al.  A study of grammatical inference , 1969 .

[4]  J. M. Barzdin,et al.  Prognostication of Automata and Functions , 1971, IFIP Congress.

[5]  Jerome A. Feldman,et al.  Some Decidability Results on Grammatical Inference and Complexity , 1972, Inf. Control..

[6]  R. V. Freivald.  Functions Computable in the Limit by Probabilistic Machines , 1974, MFCS.

[7]  Karlis Podnieks.  Comparing various concepts of function prediction. Part 1. , 1974 .

[8]  R. V. Freivald.  Minimal Gödel Numbers and Their Identification in the Limit , 1975, MFCS.

[9]  Manuel Blum,et al.  Toward a Mathematical Theory of Inductive Inference , 1975, Inf. Control..

[10]  Eliana Minicozzi,et al.  Some Natural Properties of Strong-Identification in Inductive Inference , 1976, Theor. Comput. Sci..

[11]  Azriel Rosenfeld,et al.  Grammatical inference by hill climbing , 1976, Inf. Sci..

[12]  C. M. Cook,et al.  Grammatical inference by hill climbing , 1976, Inf. Sci..

[13]  Taylor L. Booth,et al.  Inference of Finite-State Probabilistic Grammars , 1977, IEEE Transactions on Computers.

[14]  Adrian Walker,et al.  On the Inference of Stochastic Regular Grammars , 1978, Inf. Control..

[15]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[16]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[17]  Shimon Even,et al.  Graph Algorithms , 1979 .

[18]  Brian R. Gaines.  Maryanski's Grammatical Inferencer , 1979, IEEE Transactions on Computers.

[19]  Keh-Jiann Chen.  Tradeoffs in the Inductive Inference of Nearly Minimal Size Programs , 1982, Inf. Control..

[20]  Carl H. Smith,et al.  The Power of Pluralism for Automatic Program Synthesis , 1982, JACM.

[21]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[22]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[23]  D. Osherson,et al.  A note on formal learning theory , 1982, Cognition.

[24]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[25]  Robert P. Daley.  On the Error Correcting Power of Pluralism in BC-Type Inductive Inference , 1983, Theoretical Computer Science.

[26]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[27]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[28]  Rusins Freivalds,et al.  On the Power of Probabilistic Strategies in Inductive Inference , 1984, Theor. Comput. Sci..

[29]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[30]  Carl H. Smith,et al.  Probability and Plurality for Aggregations of Learning Machines , 1987, Inf. Comput..

[31]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[32]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[33]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.