Artificial neural network implementation on a single FPGA of a pipelined on-line backpropagation

The paper describes the implementation of a systolic array for a multilayer perceptron on a Virtex XCV400 FPGA with a hardware-friendly learning algorithm. A pipelined adaptation of the on-line backpropagation algorithm is shown. Parallelism is better exploited because both forward and backward phases can be performed simultaneously. We can implement very large interconnection layers by using large Xilinx devices with embedded memories alongside the projection used in the systolic architecture. These physical and architectural features --- together with the combination of FPGA reconfiguration properties with a design flow based on generic VHDL --- create an easy, flexible, and fast method of designing a complete ANN on a single FPGA. The result offers a high degree of parallelism and fast performance.

[1]  Neil W. Bergmann,et al.  Bit-serial array implementation of a multilayer perceptron , 1993 .

[2]  Nouma Izeboudjen,et al.  Digital Implementation of Artificial Neural Networks: From VHDL Description to EPGA Implementation , 1999, IWANN.

[3]  Demessie Girma,et al.  Artificial Neural Network Implementation on a Fine-Grained FPGA , 1994, FPL.

[4]  Michael Gschwind,et al.  A Fast FPGA Implementation of a General Purpose Neuron , 1994, FPL.

[5]  Scott Hauck,et al.  The roles of FPGAs in reprogrammable systems , 1998, Proc. IEEE.

[6]  W. E. Blanz,et al.  GANGLION-a fast field-programmable gate array implementation of a connectionist classifier , 1992 .

[7]  Eduardo Sanchez,et al.  Hardware Reconfigurable Neural Networks , 1998, IPPS/SPDP Workshops.

[8]  H. C. Zeidler,et al.  On-chip backpropagation training using parallel stochastic bit streams , 1996, Proceedings of Fifth International Conference on Microelectronics for Neural Networks.

[9]  R.G. Girones,et al.  Systolic implementation of a pipelined on-line backpropagation , 1999, Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems.

[10]  Guy E. Blelloch,et al.  An implementation of network learning on the Connection Machine , 1988 .

[11]  B. Burton,et al.  Reducing the computational demands of continually online trained artificial neural networks for system identification and control of fast processes , 1994, Proceedings of 1994 IEEE Industry Applications Society Annual Meeting.

[12]  Brad Hutchings,et al.  FPGA-based stochastic neural networks-implementation , 1994, Proceedings of IEEE Workshop on FPGA's for Custom Computing Machines.

[13]  Brad Hutchings,et al.  RRANN: a hardware implementation of the backpropagation algorithm using reconfigurable FPGAs , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[14]  Mark Shand,et al.  Programmable active memories: reconfigurable systems come of age , 1996, IEEE Trans. Very Large Scale Integr. Syst..

[15]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[16]  Gérard Dreyfus,et al.  Performance analysis of a pipelined backpropagation parallel algorithm , 1993, IEEE Trans. Neural Networks.

[17]  Rafael Gadea Gironés,et al.  Forward-Backward Parallelism in On-Line Backpropagation , 1999, IWANN.