Circuit complexity for neural computation

One common model for artificial neural networks is the threshold circuit multilayer feedforward network of linear threshold gates. The authors provide a rigorous analysis of this model via a circuit complexity theoretic approach by focusing on the basic computational properties of threshold circuits with binary inputs. The model of threshold circuits is shown to be computationally more powerful than the conventional model of AND-OR logic circuits. In particular, the best known results on the depth of threshold circuits are presented in implementing common arithmetic functions such as multiplication, division, and sorting. The issues of depth-size tradeoffs are investigated by demonstrating that a small increase in the depth can significantly decrease the size required in the threshold circuit for the class of symmetric functions.<<ETX>>

[1]  T. Kailath,et al.  Computing With Almost Optimal Size Threshold Circuits , 1991, Proceedings. 1991 IEEE International Symposium on Information Theory.

[2]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3]  A. Yao.  Separating the polynomial-time hierarchy by oracles , 1985 .

[4]  Johan Håstad,et al.  Almost optimal lower bounds for small depth circuits , 1986, STOC '86.

[5]  Jehoshua Bruck,et al.  On the Power of Threshold Circuits with Small Weights , 1991, SIAM J. Discret. Math..

[6]  Uzi Vishkin,et al.  Constant Depth Reducibility , 1984, SIAM J. Comput..

[7]  Jehoshua Bruck,et al.  Neural computation of arithmetic functions , 1990 .

[8]  Thomas Kailath,et al.  Depth-Size Tradeoffs for Neural Computation , 1991, IEEE Trans. Computers.

[9]  Stephen A. Cook,et al.  Log Depth Circuits for Division and Related Problems , 1986, SIAM J. Comput..

[10]  J. Reif,et al.  On Threshold Circuits and Polynomial Computation , 1992, SIAM J. Comput..

[11]  Jehoshua Bruck,et al.  Depth efficient neural networks for division and related problems , 1993, IEEE Trans. Inf. Theory.

[12]  Robert C. Minnick,et al.  Linear-Input Logic , 1961, IRE Trans. Electron. Comput..