Optimal Simulation of Automata by Neural Nets

The problem of simulation of automata by neural networks is investigated. In the case of discrete networks with polynomially bounded weights, the optimal lower and upper bounds for the number of neurons necessary to simulate any finite automata of size n are presented. For the analog case we prove the 15-neuron upper bound for any finite automaton. By extending this construction we show that a 25-neuron network may simulate any Turing machine, and hence its behavior is undecidable.

[1]  Eduardo D. Sontag,et al.  Finiteness results for sigmoidal “neural” networks , 1993, STOC.

[2]  Martin Hasler,et al.  Recursive neural networks for associative memory , 1990, Wiley-interscience series in systems and optimization.

[3]  Michel Cosnard,et al.  Computability Properties of Low-dimensional Dynamical Systems , 1993, STACS.

[4]  Hava T. Siegelmann,et al.  On the computational power of neural nets , 1992, COLT '92.

[5]  Torben Hagerup,et al.  A Guided Tour of Chernoff Bounds , 1990, Inf. Process. Lett..

[6]  Wolfgang Maass,et al.  Bounds for the computational power and learning complexity of analog neural nets , 1993, SIAM J. Comput..

[7]  Pekka Orponen,et al.  On the Computational Power of Discrete Hopfield Nets , 1993, ICALP.

[8]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[9]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[10]  Ranan B. Banerji.  Formal techniques in artificial intelligence - a sourcebook , 1990, Studies in computer science and artificial intelligence.

[11]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[12]  Noga Alon,et al.  Efficient simulation of finite automata by neural nets , 1991, JACM.

[13]  Georg Schnitger,et al.  On the computational power of sigmoid versus Boolean threshold circuits , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[14]  Pekka Orponen,et al.  Neural Networks and Complexity Theory , 1992, Nord. J. Comput..