论文信息 - Hill climbing in recurrent neural networks for learning the a/sup n/b/sup n/c/sup n/ language

Hill climbing in recurrent neural networks for learning the a/sup n/b/sup n/c/sup n/ language

A simple recurrent neural network is trained on a one-step look ahead prediction task for symbol sequences of the context-sensitive a/sup n/b/sup n/c/sup n/ language. Using an evolutionary hill climbing strategy for incremental learning the network learns to predict sequences of strings up to depth n=12. Experiments and the algorithms used are described. The activation of the hidden units of the trained network is displayed in a 3D graph and analysed.

Stephan K. Chalup | A. D. Blair | A. Blair | S. Chalup

[1] Stephan K. Chalup,et al. A study on hill climbing algorithms for neural network training , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[2] C. Lee Giles,et al. Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks , 1992, Neural Computation.

[3] Paul Rodríguez,et al. A Recurrent Neural Network that Learns to Count , 1999, Connect. Sci..

[4] Helko Lehmann,et al. Designing a Counter: Another Case Study of Dynamics and Activation Landscapes in Recurrent Networks , 1997, KI.

[5] Pekka Orponen,et al. On the Effect of Analog Noise in Discrete-Time Analog Computations , 1996, Neural Computation.

[6] J. Elman. Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[7] Jeffrey L. Elman,et al. Finding Structure in Time , 1990, Cogn. Sci..

[8] Geoffrey E. Hinton,et al. Learning representations by back-propagating errors , 1986, Nature.

[9] Mike Casey,et al. The Dynamics of Discrete-Time Computation, with Application to Recurrent Neural Networks and Finite State Machine Extraction , 1996, Neural Computation.