Neural networks, orientations of the hypercube, and algebraic threshold functions

A class of possible generalizations of current neural networks models is described using local improvement algorithms and orientations of graphs. A notation of dynamical capacity is defined and, by computing bounds on the number of algebraic threshold functions, it is proven that for neural networks of size n and energy function of degree d, this capacity is O(n/sup d+1/). Stable states are studied, and it is shown that for the same networks the storage capacity is O(n/sup d+1/). In the case of random orientations, it is proven that the expected number of stable states is exponential. Applications to coding theory are indicated, and it is shown that usual codes can be embedded in neural networks but only at high cost. Cycles and their storage are also examined. >

[1]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[2]  Saburo Muroga,et al.  Lower Bounds of the Number of Threshold Functions and a Maximum Weight , 1962, IEEE Trans. Electron. Comput..

[3]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[4]  C. Lee Giles,et al.  Nonlinear dynamics of artificial neural systems , 1987 .

[5]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[6]  Yaser S. Abu-Mostafa,et al.  Information capacity of the Hopfield model , 1985, IEEE Trans. Inf. Theory.

[7]  P. Baldi,et al.  Asymptotic normality of some Graph-Related statistics , 1989 .

[8]  Baldi,et al.  Number of stable points for spin-glasses and neural networks of higher orders. , 1987, Physical review letters.

[9]  Demetri Psaltis,et al.  Nonlinear discriminant functions and associative memories , 1987 .

[10]  Robert O. Winder,et al.  Bounds on Threshold Gate Realizability , 1963, IEEE Transactions on Electronic Computers.

[11]  Isabelle Guyon,et al.  Designing a neural network satisfying a given set of constraints , 1987 .

[12]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[14]  Richard P. Stanley.  Acyclic orientations of graphs , 1973, Discret. Math..

[15]  Saburo Muroga,et al.  Lower Bound of the Number of Threshold Functions , 1966, IEEE Trans. Electron. Comput..

[16]  John C. Platt,et al.  Analog decoding using neural networks , 1987 .

[17]  Pierre Baldi,et al.  Group Actions and Learning for a Family of Automata , 1988, J. Comput. Syst. Sci..

[18]  Shen Lin.  Computer solutions of the traveling salesman problem , 1965 .