Neural Networks: Architectures, Learning and Performance

We review recent work on some principles of a theory of neural networks. Contributions about network architectures, performance criteria, learning objectives and learning strategies from statistical mechanics, computer sciences, statistics and approximation theory are presented.

[1]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.

[2]  R Kree,et al.  Recognition of topological features of graphs and images in neural networks , 1988 .

[3]  Kree,et al.  Continuous-time dynamics of asymmetrically diluted neural networks. , 1987, Physical review. A, General physics.

[4]  Gintaras V. Reklaitis,et al.  Nonlinear Optimization Using Generalized Hopfield Networks , 1989, Neural Computation.

[5]  W. Little.  The existence of persistent states in the brain , 1974 .

[6]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[7]  P. Carnevali,et al.  Exhaustive Thermodynamical Analysis of Boolean Learning Networks , 1987 .

[8]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[9]  Giorgio Parisi,et al.  On the classification of learning machines , 1992 .

[10]  Yaser S. Abu-Mostafa,et al.  The Vapnik-Chervonenkis Dimension: Information versus Complexity in Learning , 1989, Neural Computation.

[11]  Carsten Peterson,et al.  Parallel Distributed Approaches to Combinatorial Optimization: Benchmark Studies on Traveling Salesman Problem , 1990, Neural Computation.

[12]  Griff L. Bilbro,et al.  Maximum Entropy and Learning Theory , 1992, Neural Computation.

[13]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[14]  H. Horner.  Dynamics of learning for the binary perceptron problem , 1992 .

[15]  Shun-ichi Amari,et al.  Four Types of Learning Curves , 1992, Neural Computation.

[16]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[17]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[18]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[20]  Györgyi,et al.  Inference of a rule by a neural network with thermal noise. , 1990, Physical review letters.

[21]  Richard Lippmann,et al.  Review of Neural Networks for Speech Recognition , 1989, Neural Computation.

[22]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[23]  Esther Levin,et al.  A statistical approach to learning and generalization in layered neural networks , 1989, Proc. IEEE.

[24]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[25]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[26]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[27]  M. Zak.  Terminal attractors for addressable memory in neural networks , 1988 .

[28]  Eytan Domany,et al.  Models of Neural Networks I , 1991 .

[29]  Halbert White,et al.  Learning in Artificial Neural Networks: A Statistical Perspective , 1989, Neural Computation.

[30]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[32]  J. Stephen Judd,et al.  On the complexity of loading shallow neural networks , 1988, J. Complex..

[33]  Pierre Roussel-Ragot,et al.  Neural Networks and Nonlinear Adaptive Filtering: Unifying Concepts and New Algorithms , 1993, Neural Computation.

[34]  Opper,et al.  Generalization performance of Bayes optimal classification algorithm for learning a perceptron. , 1991, Physical review letters.

[35]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[36]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[37]  A. Müller,et al.  Classification Properties of Communicating Neural Networks , 1993 .

[38]  T. Watkin,et al.  THE STATISTICAL-MECHANICS OF LEARNING A RULE , 1993 .

[39]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[40]  Sompolinsky,et al.  Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[41]  M. Golea,et al.  A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons , 1990 .

[42]  T. Watkin.  Optimal Learning with a Neural Network , 1993 .

[43]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.

[44]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[45]  D. Sherrington.  Neural networks: the spin glass approach , 1993 .

[46]  William G. Baxt,et al.  Improving the Accuracy of an Artificial Neural Network Using Multiple Differently Trained Networks , 1992, Neural Computation.

[47]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.