论文信息 - DAPHNE: DATA PARALLELISM NEURAL NETWORK SIMULATOR

DAPHNE: DATA PARALLELISM NEURAL NETWORK SIMULATOR

In this paper we describe the guideline of Daphne, a parallel simulator for supervised recurrent neural networks trained by Backpropagation through time. The simulator has a modular structure, based on a parallel training kernel running on the CM-2 Connection Machine. The training kernel is written in CM Fortran in order to exploit some advantages of the slicewise execution model. The other modules are written in serial C code. They are used for designing and testing the network, and for interfacing with the training data. A dedicated language is available for defining the network architecture, which allows the use of linked modules. The implementation of the learning procedures is based on training example parallelism. This dimension of parallelism has been found to be effective for learning static patterns using feedforward networks. We extend training example parallelism for learning sequences with full recurrent networks. Daphne is mainly conceived for applications in the field of Automatic Speech Rec...

[1] Kenneth Steiglitz,et al. Combinatorial Optimization: Algorithms and Complexity , 1981 .

[2] Alexander Singer,et al. Implementations of artificial neural networks on the Connection Machine , 1990, Parallel Comput..

[3] Bertil Svensson,et al. Using and Designing Massively Parallel Computers for Artificial Neural Neural Networks , 1992, J. Parallel Distributed Comput..

[4] Giovanni Soda,et al. An unified approach for integrating explicit knowledge and learning by example in recurrent networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[5] P. Frasconi,et al. Local Feedback Multi-Layered Networks , 1992 .

[6] Jill P. Mesirov,et al. An Efficient Implementation of the Back-propagation Algorithm on the Connection Machine CM-2 , 1989, NIPS.

[7] Thomas Fontaine. GRAD -CM2: A Data-Parallel Connectionist Network Simulator , 1992 .

[8] Lokendra Shastri,et al. Speech recognition using connectionist networks , 1988 .

[9] Carl Diegert. Out-of-core backpropagation , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[10] Geoffrey E. Hinton,et al. Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[11] Guy E. Blelloch,et al. An implementation of network learning on the Connection Machine , 1988 .

[12] Geoffrey E. Hinton,et al. Learning internal representations by error propagation , 1986 .