Simulation and hardware implementation of competitive learning neural networks

One of the main research topics within neuron-like networks is related to learning techniques. Competitive learning has got an special interest among them, because a great network automation is achieved with it, ie, autonomously and without explicit indication of the correct output patterns, the network extracts general features that can be used in order to cluster a set of patterns. In this paper, after giving a brief overview about learning procedures, the most peculiar characteristics of competitive learning are pointed out, and the different ways of implementing neuron-like networks are quoted, describing as an implementation instance our present project of hardware construction of a neural chip to be included in a coprocessor board with competitive learning.

[1]  R. Hecht-Nielsen,et al.  Neurocomputing: picking the human brain , 1988, IEEE Spectrum.

[2]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[3]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[4]  S. Garth.  A dedicated computer for simulation of large systems of neural nets , 1988 .

[5]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[6]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..

[7]  Luis B. Almeida.  Back propagation in non-feedforward networks , 1989 .

[8]  Shun-ichi Amari,et al.  Field theory of self-organizing neural nets , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Teuvo Kohonen,et al.  The 'neural' phonetic typewriter , 1988, Computer.

[10]  Kunihiko Fukushima,et al.  A neural network for visual pattern recognition , 1988, Computer.

[11]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[12]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[13]  Robert O. Grondin,et al.  Limited interconnectivity in synthetic neural systems , 1989 .

[14]  Lawrence D. Jackel,et al.  VLSI implementation of a neural network model , 1988, Computer.

[15]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[16]  F. J. Kub,et al.  Programmable analog vector-matrix multipliers , 1990 .

[17]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[18]  Carver A. Mead,et al.  Implementing neural architectures using analog VLSI circuits , 1989 .

[19]  Klaus Schumacher,et al.  VLSI technologies for artificial neural networks , 1989, IEEE Micro.

[20]  Michel Verleysen,et al.  An analog VLSI implementation of Hopfield's neural network , 1989, IEEE Micro.

[21]  M. Vellasco,et al.  VLSI architectures for neural networks , 1989, IEEE Micro.

[22]  R. Hecht-Nielsen,et al.  Neurocomputer applications , 1989 .

[23]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[24]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[25]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[26]  Robert B. Allen,et al.  Stochastic Learning Networks and their Electronic Implementation , 1987, NIPS.

[27]  D. B. Schwartz,et al.  Microelectronic Implementations of Connectionist Neural Networks , 1987, NIPS.

[28]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[29]  Terrence J. Sejnowski,et al.  Neural network learning algorithms , 1988 .

[30]  V. Vemuri,et al.  Artificial neural networks: an introduction , 1988 .

[31]  Geoffrey E. Hinton.  20 – CONNECTIONIST LEARNING PROCEDURES1 , 1990 .

[32]  S. Mackie.  Implementation of neural network models in silicon , 1989 .

[33]  Richard Lippmann,et al.  Review of Neural Networks for Speech Recognition , 1989, Neural Computation.

[34]  Michel Verleysen,et al.  Neural networks for high-storage content-addressable memory: VLSI circuit and learning algorithm , 1989 .

[35]  I. Aleksander.  A review of parallel distributed processing , 1989 .

[36]  Steven B. Bibyk,et al.  Programmable current-mode neural network for implementation in analogue MOS VLSI , 1990 .

[37]  J.J. Hopfield,et al.  Artificial neural networks , 1988, IEEE Circuits and Devices Magazine.

[38]  R. Eckmiller,et al.  Neural Computers , 1989, Springer Study Edition.

[39]  S. Kaplan.  The Physiology of Thought , 1950 .