Realizing general MLP networks with minimal FPGA resources

In recent years, there has been significant interest in implementing neural networks on FPGAs. This paper describes a simple technique for implementing multi-layer neural networks, with arbitrary numbers of neurons and layers, on FPGAs, using minimal resources. The network architecture can be modified simply by loading memory with the architecture parameters and the network weights and biases. The paper also presents an application of the technology, in which a smart position sensor system is implemented with a neural network on a Xilinx Spartan 3E FPGA development system.

[1]  Myung Won Kim,et al.  The effect of initial weights on premature saturation in back-propagation learning , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[2]  Martin T. Hagan,et al.  Gauss-Newton approximation to Bayesian learning , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[3]  Marvin Minsky,et al.  Perceptrons: expanded edition , 1988 .

[4]  Miguel A. Vega-Rodríguez,et al.  Using FPGAs to Implement Artificial Neural Networks , 2006, International Conference on Electronics, Circuits, and Systems.

[5]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[6]  Martin T. Hagan,et al.  Neural network design , 1995 .

[7]  Vincenzo Conti,et al.  Efficient MLP digital implementation on FPGA , 2005, 8th Euromicro Conference on Digital System Design (DSD'05).

[8]  Shawki Areibi,et al.  The Impact of Arithmetic Representation on Implementing MLP-BP on FPGAs: A Study , 2007, IEEE Transactions on Neural Networks.

[9]  Shawki Areibi,et al.  Arithmetic formats for implementing artificial neural networks on FPGAs , 2006, Canadian Journal of Electrical and Computer Engineering.

[10]  Brad Hutchings,et al.  RRANN: a hardware implementation of the backpropagation algorithm using reconfigurable FPGAs , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[11]  D. E. Rumelhart,et al.  chapter Parallel Distributed Processing, Exploration in the Microstructure of Cognition , 1986 .

[12]  Antonio Cañas,et al.  FPGA Implementation of a Perceptron-Like Neural Network for Embedded Applications , 2003, IWANN.

[13]  Steven A. Guccione,et al.  Neural network implementation using reconfigurable architectures , 1994 .

[14]  J.M. Granado,et al.  Using FPGAs to Implement Artificial Neural Networks , 2006, 2006 13th IEEE International Conference on Electronics, Circuits and Systems.

[15]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[16]  Nadia Nedjah,et al.  Reconfigurable Hardware Architecture for Compact and Efficient Stochastic Neuron , 2003, IWANN.