Dynamics of learning for the binary perceptron problem

A polynomial learning algorithm for a perceptron with binary bonds and random patterns is investigated within dynamic mean field theory. A discontinuous freezing transition is found at a temperature where the entropy is still positive. Critical slowing down is observed approaching this temperature from above. The fraction of errors resulting from this learning procedure is finite in the thermodynamic limit for all temperatures and all finite values of the number of patterns per bond. Monte-Carlo simulations on larger samples (N≧127) are in quantitative agreement. Simulations on smaller samples indicate a finite bound for the existence of perfect solutions in agreement with the replica theory and the zero entropy criterion. This suggests that perfect solutions exist also in larger samples but cannot be found with a polynomial procedure as expected for a combinatorial hard problem.

[1]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[2]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[3]  C. Dominicis.  Dynamics as a substitute for replicas in systems with quenched random impurities , 1978 .

[4]  D. Thouless,et al.  Stability of the Sherrington-Kirkpatrick solution of a spin glass model , 1978 .

[5]  Giorgio Parisi,et al.  Infinite Number of Order Parameters for Spin-Glasses , 1979 .

[6]  B. Derrida.  Random-energy model: An exactly solvable model of disordered systems , 1981 .

[7]  H. Sompolinsky,et al.  Relaxational dynamics of the Edwards-Anderson model and the mean-field theory of spin-glasses , 1982 .

[8]  M. Mézard,et al.  The simplest spin glass , 1984 .

[9]  H. Horner.  Dynamic mean field theory of the SK-spin glass , 1984 .

[10]  H. Horner.  Dynamic mean field theory of the SK-spin glass , 1984 .

[11]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[12]  Sommers.  Path-integral approach to Ising spin-glass dynamics. , 1987, Physical review letters.

[13]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[14]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[15]  Kree,et al.  Continuous-time dynamics of asymmetrically diluted neural networks. , 1987, Physical review. A, General physics.

[16]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[17]  E. Gardner,et al.  An Exactly Solvable Asymmetric Neural Network Model , 1987 .

[18]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[19]  Glauber Dynamics of Neural Network Models , 1988 .

[20]  E. Gardner.  The space of interactions in neural network models , 1988 .

[21]  H. Horner,et al.  Transients and basins of attraction in neutral network models , 1989 .

[22]  E. Gardner,et al.  Three unfinished works on the optimal storage capacity of networks , 1989 .

[23]  Michael Biehl,et al.  The AdaTron: An Adaptive Perceptron Algorithm , 1989 .

[24]  Werner Krauth,et al.  Critical storage capacity of the J = ± 1 neural network , 1989 .

[25]  Györgyi,et al.  First-order transition to perfect generalization in a neural network with binary synapses. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[26]  H M Kohler,et al.  Adaptive genetic algorithm for the binary perceptron problem , 1990 .

[27]  H. Gutfreund,et al.  Capacity of neural networks with discrete synaptic couplings , 1990 .

[28]  Anthony N. Burkitt,et al.  Computer-Simulation Methods , 1990 .

[29]  Properties of an adiabatically cooled SK spin glass , 1990 .

[30]  H. Horner,et al.  Dynamics of the finite SK spin-glass , 1991 .