Reductions among prediction problems: on the difficulty of predicting automata

Given examples of words accepted and rejected by an unknown automaton, the question of whether there is an algorithm that in a feasible amount of time will learn to predict which words will be accepted by the automaton is examined. A notion of prediction-preserving reducibility is developed, and it is shown that if DFAs are predictable, then so are all languages in logspace. In particular, the predictability of DFAs implies the predictability of all Booleman formulas. Similar results hold for NFAs and PDAs (or CFGs). Relationships between the complexity of the membership problem for a class of automata and the complexity of the prediction problem are obtained. Examples are given of prediction problems in which predictability implies the predictability of all languages in P. Assuming the existence of one-way functions, it follows that these problems are not predictable, even in an extremely weak sense.<<ETX>>

[1]  N. S. Barnett,et al.  Private communication , 1969 .

[2]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[3]  Neil D. Jones,et al.  Complete problems for deterministic polynomial time , 1974, STOC '74.

[4]  I. H. Sudborough.  On Tape-Bounded Complexity Classes and Multihead Finite Automata , 1973, J. Comput. Syst. Sci..

[5]  Neil D. Jones,et al.  Space-Bounded Reducibility among Combinatorial Problems , 1975, J. Comput. Syst. Sci..

[6]  Neil D. Jones,et al.  Complete problems for deterministic polynomial time , 1974, Symposium on the Theory of Computing.

[7]  Nancy A. Lynch,et al.  Log Space Recognition and Translation of Parenthesis Languages , 1977, JACM.

[8]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[9]  DANA ANGLUIN,et al.  On the Complexity of Minimum Inference of Regular Sets , 1978, Inf. Control..

[10]  Richard J. Lipton,et al.  Linear Programming is Log-Space Hard for P , 1979, Inf. Process. Lett..

[11]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[12]  David Haussler,et al.  Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension , 1986, STOC '86.

[13]  Silvio Micali,et al.  How to construct random functions , 1986, JACM.

[14]  David Haussler,et al.  Epsilon-nets and simplex range queries , 1986, SCG '86.

[15]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[16]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[17]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[18]  Michael A. Arbib,et al.  An Introduction to Formal Language Theory , 1988, Texts and Monographs in Computer Science.

[19]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[20]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.