The Science of Making ERORS: What Error Tolerance Implies for Capacity in Neural Networks

Discusses the development of formal protocols for handling error tolerance which allow a precise determination of the computational gains that may be expected. The error protocols are illustrated in the framework of a densely interconnected neural network architecture (with associative memory the putative application), and rigorous calculations of capacity ar shown. Explicit capacities are also derived for the case of feedforward neural network configurations. >

[1]  J. G. Wendel.  A Problem in Geometric Probability. , 1962 .

[2]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[3]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[4]  Robert J. McEliece,et al.  The Theory of Information and Coding , 1979 .

[5]  Charles M. Newman,et al.  Memory capacity in neural network models: Rigorous lower bounds , 1988, Neural Networks.

[6]  János Komlós,et al.  Convergence results in an associative memory model , 1988, Neural Networks.

[7]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[8]  Pierre Baldi,et al.  Programmed interactions in higher-order neural networks: Maximal capacity , 1991, J. Complex..

[9]  Ian F. Blake,et al.  Approximations for the probability in the tails of the binomial distribution , 1987, IEEE Trans. Inf. Theory.

[10]  S. Venkatesh.  Epsilon capacity of neural networks , 1987 .

[11]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[12]  S. Venkatesh.  Computation and learning in the context of neural network capacity , 1992 .

[13]  H. Schwarz.  Gesammelte mathematische Abhandlungen , 1970 .

[14]  Z. Fiiredi.  Random Polytopes in the d-Dimensional Cube , 1986 .

[15]  Zoltán Füredi.  Random polytopes in thed-dimensional cube , 1986, Discret. Comput. Geom..

[16]  Eric B. Baum,et al.  On the capabilities of multilayer perceptrons , 1988, J. Complex..