Digital Hardware Implementation of a Neural System Used for Nonlinear Adaptive Prediction

Neural networks have been widely used for many applications in digital communications. They are able to give solutions to complex problems due to their nonlinear processing and their learning and generalization. Neural networks are one of the key technologies for the communication domain and accordingly a special effort may be expected to be paid to real time hardware implementation issues. In this study, it is proposed a digital hardware implementation of a neural system based on a multilayer perceptron (MLP). The neural system is used for the nonlinear adaptive prediction of nonstationary signals such as speech signals. The implemented architecture of the MLP is generated using a generic elementary neuron (EN). The polynomial approximation method is used to implement the sigmoidal activation function. The back-propagation algorithm is used to implant the prediction task. The circuit implementation architecture is detailed, for achieving real-time prediction for speech signals. The designed ASIC circuit includes a neural network block, an on-chip learning block and a memory used for storing the synaptic weights for updating.

[1]  Michael John Sebastian Smith,et al.  Application-specific integrated circuits , 1997 .

[2]  K. M. Curtis,et al.  Piecewise linear approximation applied to nonlinear function of a neural network , 1997 .

[3]  Eduardo D. Sontag,et al.  Neural Networks for Control , 1993 .

[4]  Fernando Morgado Dias,et al.  Artificial neural networks: a review of commercial hardware , 2004, Eng. Appl. Artif. Intell..

[5]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[6]  J L Castro,et al.  Neural networks with a continuous squashing function in the output are universal approximators , 2000, Neural Networks.

[7]  Xiao Zhi Gao,et al.  Power prediction in mobile communication systems using an optimal neural-network structure , 1997, IEEE Trans. Neural Networks.

[8]  Dan Hammerstrom A Digital VLSI Architecture for Real-World Applications , 1999 .

[9]  Liang Li,et al.  Nonlinear adaptive prediction of nonstationary signals , 1995, IEEE Trans. Signal Process..

[10]  J. Beichter,et al.  Design of a 1st Generation Neurocomputer , 1991 .

[11]  Joel L. Davis,et al.  An Introduction to Neural and Electronic Networks , 1995 .

[12]  Hiroomi Hikawa Frequency-based multilayer neural network with on-chip learning and enhanced neuron characteristics , 1999, IEEE Trans. Neural Networks.

[13]  Mohamed Ibnkahla,et al.  Applications of neural networks to digital communications - a survey , 2000, Signal Process..

[14]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[15]  Matti Tommiska,et al.  Efficient digital implementation of the sigmoid function for reprogrammable logic , 2003 .

[16]  Danilo P. Mandic,et al.  Toward an optimal PRNN-based nonlinear predictor , 1999, IEEE Trans. Neural Networks.

[17]  Jean-Michel Muller,et al.  Arithmetique des Ordinateurs--Operateurs et fonctions elementaires. , 1990 .

[18]  Kamel Besbes,et al.  Digital hardware implementation of sigmoid function and its derivative for artificial neural networks , 2001, ICM 2001 Proceedings. The 13th International Conference on Microelectronics..

[19]  Mongia Mhiri,et al.  A Non-recurrent Low-Complexity Neural Network for the Speech Prediction , 2022 .

[20]  P. Laurent Approximation et optimisation , 1972 .

[21]  K. Besbes,et al.  Digital hardware implementation of a neural network used for classification , 2004, Proceedings. The 16th International Conference on Microelectronics, 2004. ICM 2004..

[22]  James L. McClelland,et al.  James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.

[23]  Stamatis Vassiliadis,et al.  Sigmoid Generators for Neural Computing Using Piecewise Approximations , 1996, IEEE Trans. Computers.

[24]  C. Alippi,et al.  Simple approximation of sigmoidal functions: realistic design of digital neural networks capable of learning , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[25]  Richard S. Sutton,et al.  Neural networks for control , 1990 .

[26]  Koldo Basterrextea,et al.  Approximation of Sigmoid Function and the Derivative for Artificial Neurons , 2000 .

[27]  D. J. Myers,et al.  Efficient implementation of piecewise linear activation function for digital VLSI neural networks , 1989 .

[28]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[29]  Ralph Judson Smith,et al.  Circuits, devices and systems , 1966 .

[30]  Robert A. Walker,et al.  Introduction to the Scheduling Problem , 1995, IEEE Des. Test Comput..

[31]  Yusuf Leblebici,et al.  Mixed analogue-digital artificial-neural-network architecture with on-chip learning , 1999 .

[32]  Himanshu Bhathagar Advanced ASIC Chip Synthesis , 1999 .

[33]  J. M. Tarela,et al.  Digital design of sigmoid approximator for artificial neural networks , 2002 .