Competitive Learning Algorithms and Neurocomputer Architecture

This paper begins with an overview of several competitive learning algorithms in artificial neural networks, including self-organizing feature maps, focusing on properties of these algorithms important to hardware implementations. We then discuss previously reported digital implementations of these networks. Finally, we report a reconfigurable parallel neurocomputer architecture we have designed using digital signal processing chips and field-programmable gate array devices. Communications are based upon a broadcast network with FPGA-based message preprocessing and postprocessing. A small prototype of this system has been constructed and applied to competitive learning in self-organizing maps. This machine is able to model slowly-varying nonstationary data in real time.

[1]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[2]  Reinhard Männer,et al.  Multiprocessor And Memory Architecture Of The Neurocomputer Synapse-1 , 1993, Int. J. Neural Syst..

[3]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[4]  R. O. Grondin,et al.  VLSI Implementation of Neural Classifiers , 1990, Neural Computation.

[5]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[6]  Manfred R. Schroeder,et al.  Code-excited linear prediction(CELP): High-quality speech at very low bit rates , 1985, ICASSP '85. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  Wolfgang Rosenstiel,et al.  Coprocessors for special neural networks: KOKOS and KOBOLD , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[8]  M. A. Jabri,et al.  Adaptive Analog VLSI Neural Systems , 1995, Springer Netherlands.

[9]  Bing J. Sheu,et al.  Neural information processing and VLSI , 1995 .

[10]  Brian R. Gaines,et al.  Stochastic Computing Systems , 1969 .

[11]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[12]  Stanley C. Ahalt,et al.  Neural Networks for Vector Quantization of Speech and Images , 1990, IEEE J. Sel. Areas Commun..

[13]  Alan F. Murray,et al.  Pulse-stream VLSI neural networks mixing analog and digital techniques , 1991, IEEE Trans. Neural Networks.

[14]  Patrick Thiran,et al.  Quantization effects in digitally behaving circuit implementations of Kohonen networks , 1994, IEEE Trans. Neural Networks.

[15]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[16]  H. T. Kung.  Why systolic architectures? , 1982, Computer.

[17]  Jenq-Neng Hwang,et al.  A unifying algorithm/architecture for artificial neural networks , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[18]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[19]  Kimmo Kaski,et al.  Pulse-density modulation technique in VLSI implementations of neural network algorithms , 1990 .

[20]  Howard C. Card.  Analog Circuits For Relaxation Networks , 1993, Int. J. Neural Syst..

[21]  Christoph von der Malsburg,et al.  Network self-organization , 1990 .

[22]  Reinhard Männer,et al.  Multiprocessor And Memory Architecture Of The Neurocomputer Synapse-1 , 1993, Int. J. Neural Syst..

[23]  Robert D. McLeod,et al.  Polymorphic computing paradigms realized for a fpd-based multicomputer , 1995 .

[24]  Stuart Mackie,et al.  A digital implementation of a best match classifier , 1988, Proceedings of the IEEE 1988 Custom Integrated Circuits Conference.

[25]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[26]  Duane DeSieno,et al.  Adding a conscience to competitive learning , 1988, IEEE 1988 International Conference on Neural Networks.

[27]  K. Wojtek Przytula.  Parallel digital implementations of neural networks , 1991, ASAP.

[28]  Max Stanford Tomlinson,et al.  A digital neural network architecture for VLSI , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[29]  Howard Card,et al.  Digital VLSI backpropagation networks , 1995, Canadian Journal of Electrical and Computer Engineering.

[30]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[31]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[32]  J. Bailey,et al.  Why VLSI implementations of associative VLCNs require connection multiplexing , 1988, IEEE 1988 International Conference on Neural Networks.

[33]  Sun-Yuan Kung,et al.  Digital neural networks , 1993, Prentice Hall Information and System Sciences Series.

[34]  Wolfgang Rosenstiel,et al.  A hardware supported system for a special online application of self-organizing map , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[35]  Douglas S. Reeves,et al.  The TInMANN VLSI chip , 1992, IEEE Trans. Neural Networks.

[36]  S. Grossberg,et al.  A self-organizing neural network for supervised learning, recognition, and prediction , 1992, IEEE Communications Magazine.

[37]  W. E. Blanz,et al.  GANGLION-a fast field-programmable gate array implementation of a connectionist classifier , 1992 .