Finite-size effects and bounds for perceptron models

AblncL In lhis paper we consider two main aspsln of the binary perccptron problem: the maximal capacity when random paltem are stored (model A), and its generalhation ability (model B). We have extended previous numerical estimates of critical capacilies and studied thermal properties of systems of small sizes to lest recent replica predictions. We have also considered some simpler versions of thest models. The discrete spherical vemions can be salved exactly using Gardner’s replica calculation for the spherical model and are shown to give a rigorous upper bound and lower bound on the capacities of models A and B. respectively. Iby vemions of models A and B are soivcd in detail and provide information which is useful for interpreting the Bnile-size eEeciects present in the numerical studies of models A and B.

[1]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[2]  Bernard Derrida,et al.  Finite-size effects in random energy models and in the problem of polymers in a random medium , 1991 .

[3]  H. Gutfreund,et al.  Learning and retrieval in attractor neural networks above saturation , 1991 .

[4]  I. Kanter,et al.  Storage Capacity of a Multilayer Neural Network with Binary Weights , 1991 .

[5]  H M Kohler,et al.  Adaptive genetic algorithm for the binary perceptron problem , 1990 .

[6]  Optimally adapted attractor neural networks in the presence of noise , 1990 .

[7]  Sompolinsky,et al.  Learning from examples in large neural networks. , 1990, Physical review letters.

[8]  D. Amit,et al.  Retrieval phase diagrams for attractor neural networks with optimal interactions , 1990 .

[9]  José F. Fontanari,et al.  Landscape statistics of the binary perceptron , 1990 .

[10]  H. Gutfreund,et al.  Capacity of neural networks with discrete synaptic couplings , 1990 .

[11]  M. Opper,et al.  On the ability of the optimal perceptron to generalise , 1990 .

[12]  Györgyi,et al.  First-order transition to perfect generalization in a neural network with binary synapses. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[13]  Haim Sompolinsky,et al.  Learning from Examples in a Single-Layer Neural Network , 1990 .

[14]  Miguel Angel Virasoro,et al.  Prosopagnosia in high capacity neural networks storing uncorrelated classes , 1990 .

[15]  D. Sherrington,et al.  The Optimal Retrieval in Boolean Neural Networks , 1989 .

[16]  W. Krauth,et al.  Storage capacity of memory networks with binary couplings , 1989 .

[17]  Edoardo Amaldi,et al.  Stability-Capacity Diagram of a Neural Network with Ising Bonds , 1989 .

[18]  R. Meir,et al.  Mapping correlated Gaussian patterns in a perceptron , 1989 .

[19]  I. Kondor,et al.  Spin-glass field theory in the condensed phase continued to below d =6 , 1989 .

[20]  E. Gardner,et al.  Three unfinished works on the optimal storage capacity of networks , 1989 .

[21]  E Gardner.  Optimal basins of attraction in randomly sparse neural network models , 1989 .

[22]  H. Gutfreund,et al.  The phase space of interactions in neural networks with definite symmetry , 1989 .

[23]  Thomas B. Kepler,et al.  Universality in the space of interactions for network models , 1989 .

[24]  F. Vallet,et al.  Linear and Nonlinear Extension of the Pseudo-Inverse Solution for Learning Boolean Functions , 1989 .

[25]  Werner Krauth,et al.  Critical storage capacity of the J = ± 1 neural network , 1989 .

[26]  F. Vallet.  The Hebb Rule for Learning Linearly Separable Boolean Functions: Learning and Generalization , 1989 .

[27]  M. Virasoro,et al.  Perceptron beyond the limit of capacity , 1989 .

[28]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[29]  E. Gardner.  The space of interactions in neural network models , 1988 .

[30]  E. Gardner,et al.  Maximum Storage Capacity in Neural Networks , 1987 .

[31]  D. Ruelle.  A mathematical reformulation of Derrida's REM and GREM , 1987 .

[32]  M. Mézard,et al.  The simplest spin glass , 1984 .

[33]  A. Bray,et al.  Lack of self-averaging in spin glasses , 1984 .

[34]  B. Derrida.  Random-energy model: An exactly solvable model of disordered systems , 1981 .

[35]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..