Software for neural networks

Neural networks "compute" though not in the way that traditional computers do. It is necessary to accept their weaknesses to use their strengths. We discuss some of the assumptions and constraints that govern operation of neural nets, describe one particular non-linear network---the BSB model---in a little detail, and present two applications of neural network computations to illustrate some of the peculiarities inherent in this architecture. We show how a network can be trained to estimate answers to simple multiplication problems and how a network can be used to disambiguate lexical items by context. In both examples, the way information is represented in the pattern of system unit activities is at least as important as the details of the learning and retrieval algorithms used.

[1]  Teuvo Kohonen,et al.  Associative memory. A system-theoretical approach , 1977 .

[2]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[3]  Joyce L Moore.  Back-of-the-Envelope Problems. , 1987 .

[4]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[5]  Massimo A. Sivilotti,et al.  Real-time visual computations using analog CMOS processing arrays , 1987 .

[6]  R. Golden.  The :20Brain-state-in-a-box Neural model is a gradient descent algorithm , 1986 .

[7]  Jamie I. D. Campbell,et al.  Mental multiplication skill: Structure, process, and acquisition. , 1985 .

[8]  J. Hadamard,et al.  The Psychology of Invention in the Mathematical Field. , 1945 .

[9]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Jacques Hadamard,et al.  The Psychology of Invention in the Mathematical Field. , 1949 .

[11]  J. A. Anderson,et al.  Associative learning of scene parameters from images. , 1987, Applied optics.

[12]  James A. Anderson,et al.  Cognitive and psychological computation with neural models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  P. J. Davis,et al.  Nonanalytic aspects on mathematics and their implication on research and education , 1979 .

[14]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[15]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[16]  L D Jackel,et al.  Electronic neural network chips. , 1987, Applied optics.

[17]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[18]  Gregory L. Murphy,et al.  Psychological concepts in a parallel system , 1986 .

[19]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.