Quenched versus annealed dilution in neural networks

The capacity for storing random patterns in a diluted neural network is determined following the method of Gardner. Two types of dilution are considered. In the quenched case, the broken couplings are chosen at random and are independent of the stored patterns. By contrast, in the annealed case, the disconnected couplings are selected in order to optimize the storage of the patterns. By the same token, the vanishing couplings are strongly correlated with the stored patterns. We also determine the distribution of the synaptic strengths. This distribution illustrates the difference between quenched and annealed dilution most clearly.

[1]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[2]  van Aernout Enter,et al.  Chopper model of pattern recognition. , 1986 .

[3]  Sompolinsky,et al.  Neural networks with nonlinear synapses and a static noise. , 1986, Physical review. A, General physics.

[4]  J. Hemmen.  Nonlinear neural networks near saturation. , 1987 .

[5]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[6]  E. Gardner,et al.  Zero temperature parallel dynamics for infinite range spin glasses and neural networks , 1987 .

[7]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[8]  Marc Mézard,et al.  The roles of stability and symmetry in the dynamics of neural networks , 1988 .

[9]  M. A. Virasoro,et al.  The Effect of Synapses Destruction on Categorization by Neural Networks , 1988 .

[10]  Andrew Canning,et al.  Partially connected models of neural networks , 1988 .

[11]  E. Gardner.  The space of interactions in neural network models , 1988 .

[12]  B. Forrest.  Content-addressability and learning in neural networks , 1988 .

[13]  Thomas B. Kepler,et al.  Domains of attraction in neural networks , 1988 .

[14]  Noest.  Domains in neural networks with restricted-range interactions. , 1989, Physical review letters.

[15]  E Gardner.  Optimal basins of attraction in randomly sparse neural network models , 1989 .

[16]  D. Amit,et al.  Retrieval phase diagrams for attractor neural networks with optimal interactions , 1990 .

[17]  Jean-Pierre Nadal,et al.  Information storage in sparsely coded memory nets , 1990 .

[18]  M Bouten,et al.  Storage capacity of a diluted neural network with Ising couplings , 1990 .