Analog Stable Simulation of Discrete Neural Networks

The nite discrete time recurrent neural networks are also exploited for potentially in nite computations e g nite automata where the input is being gradually pre sented from an external environment via input neurons Because of gradient learning heuristics or analog hardware implementation reasons the usage of some continuous ac tivation function is sometimes preferred rather than the discrete hard limiter threshold function However in such cases the approximate representation of nite automaton states by analog network states can lead to an unstable behavior for long input se quences and consequently to an incorrect resulting computation Therefore a stable simulation of any discrete neural network by an analog network of the same size is proposed The simulation works in real time step per step for any real activation function with di erent nite limits in improper points In fact only the weight pa rameters of the analog neural network are adjusted to achieve su cient state precision The same result holds for symmetric networks

[1]  Padhraic Smyth,et al.  Learning Finite State Machines With Self-Clustering Recurrent Networks , 1993, Neural Computation.

[2]  Noga Alon,et al.  Efficient simulation of finite automata by neural nets , 1991, JACM.

[3]  Giovanni Soda,et al.  An unified approach for integrating explicit knowledge and learning by example in recurrent networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[4]  Pekka Orponen,et al.  The Computational Power of Discrete Hopfield Nets with Hidden Units , 1996, Neural Computation.

[5]  C. Lee Giles,et al.  Constructing deterministic finite-state automata in recurrent neural networks , 1996, JACM.

[6]  Don R. Hush,et al.  Bounds on the complexity of recurrent neural network implementations of finite state machines , 1993, Neural Networks.

[7]  Eduardo Sontag,et al.  Computational power of neural networks , 1995 .

[8]  C. Lee Giles,et al.  Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks , 1992, Neural Computation.

[9]  Hava T. Siegelmann,et al.  Analog computation via neural networks , 1993, [1993] The 2nd Israel Symposium on Theory and Computing Systems.

[10]  Piotr Indyk.  Optimal Simulation of Automata by Neural Nets , 1995, STACS.

[11]  S C Kleene,et al.  Representation of Events in Nerve Nets and Finite Automata , 1951 .

[12]  Pekka Orponen,et al.  On the Effect of Analog Noise in Discrete-Time Analog Computations , 1996, Neural Computation.

[13]  Panagiotis Manolios,et al.  First-Order Recurrent Neural Networks and Deterministic Finite State Automata , 1994, Neural Computation.

[14]  J. McCarthy,et al.  Automata Studies. (AM-34) , 1956 .