Learning by dilution in a neural network

A perceptron with N random weights can store of the order of N patterns by removing a fraction of the weights without changing their strengths. The critical storage capacity as a function of the concentration of the remaining bonds for random outputs and for outputs given by a teacher perceptron is calculated. A simple Hebb-like dilution algorithm is presented which, in the teacher case, reaches the optimal generalization ability.

[1]  Bouten,et al.  Replica symmetry breaking in a diluted network with binary couplings. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[2]  M. Opper,et al.  Statistical mechanics of the knapsack problem , 1994 .

[3]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[4]  R. Urbanczik.  Storage Capacity of the Tree-Structured Committee Machine with Discrete Weights , 1994 .

[5]  Désiré Bollé,et al.  Capacity of diluted multi-state neural networks , 1994 .

[6]  I. Jensen.  Critical behaviour of a surface reaction model with infinitely many absorbing states , 1993, cond-mat/9312065.

[7]  H. K. Patel,et al.  Computational complexity, learning rules and storage capacities: A Monte Carlo study for the binary perceptron , 1993 .

[8]  C. Van Den Broeck,et al.  Clipped-Hebbian Training of the Perceptron , 1993 .

[9]  T. Watkin,et al.  THE STATISTICAL-MECHANICS OF LEARNING A RULE , 1993 .

[10]  C. Peterson,et al.  Neural Networks for Optimization Problems with Inequality Constraints: The Knapsack Problem , 1993, Neural Computation.

[11]  A. Coolen,et al.  Learning in neural networks by eliminating frustrated bonds , 1993 .

[12]  H. Horner.  Dynamics of learning for the binary perceptron problem , 1992 .

[13]  P. Kuhlmann,et al.  A dilution algorithm for neural networks , 1992 .

[14]  Sompolinsky,et al.  Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[15]  Luca Vogt,et al.  Models of Neural Networks I , 1991 .

[16]  A. Komoda,et al.  Quenched versus annealed dilution in neural networks , 1990 .

[17]  H. Gutfreund,et al.  Capacity of neural networks with discrete synaptic couplings , 1990 .

[18]  Wolfgang Kinzel,et al.  Learning algorithm for a neural network with binary synapses , 1990 .

[19]  Györgyi,et al.  First-order transition to perfect generalization in a neural network with binary synapses. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[20]  Vallet,et al.  Recognition rates of the Hebb rule for learning Boolean functions. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[21]  W. Krauth,et al.  Storage capacity of memory networks with binary couplings , 1989 .

[22]  F. Vallet.  The Hebb Rule for Learning Linearly Separable Boolean Functions: Learning and Generalization , 1989 .

[23]  Andrew Canning,et al.  Partially connected models of neural networks , 1988 .

[24]  E. Gardner.  The space of interactions in neural network models , 1988 .

[25]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[26]  I. Morgenstern,et al.  Heidelberg Colloquium on Glassy Dynamics , 1987 .

[27]  van Aernout Enter,et al.  Chopper model of pattern recognition. , 1986 .

[28]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[29]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[30]  D. Thouless,et al.  Stability of the Sherrington-Kirkpatrick solution of a spin glass model , 1978 .

[31]  G. Lynch,et al.  Memory: Organization and locus of change , 1994 .