A survey of perceptron circuit complexity results

This paper surveys many circuit complexity results for networks of perceptrons, focusing on those presented over the last ten years. The first part reviews general theoretical results, while the second part is more practical, comparing nine different constructive solutions for the addition of two binary numbers.

[1]  Jehoshua Bruck,et al.  Depth efficient neural networks for division and related problems , 1993, IEEE Trans. Inf. Theory.

[2]  Michael Sipser,et al.  Parity, circuits, and the polynomial-time hierarchy , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[3]  Eric Allender,et al.  A note on the power of threshold circuits , 1989, 30th Annual Symposium on Foundations of Computer Science.

[4]  Valeriu Beiu,et al.  VLSI implementations of threshold logic-a comprehensive survey , 2003, IEEE Trans. Neural Networks.

[5]  Belle W. Y. Wei,et al.  Area-Time Optimal Adder Design , 1990, IEEE Trans. Computers.

[6]  C. Pacha,et al.  Aspects of systems and circuits for nanoelectronics , 1997, Proc. IEEE.

[7]  Thomas Hofmeister,et al.  Some Notes on Threshold Circuits, and Multiplication in Depth 4 , 1991, Inf. Process. Lett..

[8]  Alon Orlitsky,et al.  Lower bounds on threshold and related circuits via communication complexity , 1994, IEEE Trans. Inf. Theory.

[9]  H. T. Kung,et al.  A Regular Layout for Parallel Adders , 1982, IEEE Transactions on Computers.

[10]  K. Siu,et al.  Theoretical Advances in Neural Computation and Learning , 1994, Springer US.

[11]  Jehoshua Bruck,et al.  On the Power of Threshold Circuits with Small Weights , 1991, SIAM J. Discret. Math..

[12]  Marek Karpinski,et al.  Simulating threshold circuits by majority circuits , 1993, SIAM J. Comput..

[13]  Vojin G. Oklobdzija,et al.  Delay Optimization of Carry-Skip Adders and Block Carry-Lookahead Adders Using Multidimensional Dynamic Programming , 1992, IEEE Trans. Computers.

[14]  Valeriu Beiu,et al.  On the Circuit Complexity of Sigmoid Feedforward Neural Networks , 1996, Neural Networks.

[15]  Werner Prost,et al.  Threshold logic circuit design of parallel adders using resonant tunneling devices , 2000, IEEE Trans. Very Large Scale Integr. Syst..

[16]  Daniel E. Atkins,et al.  A comparison of ALU structures for VLSI technology , 1983, 1983 IEEE 6th Symposium on Computer Arithmetic (ARITH).

[17]  Nicholas Pippenger.  The Complexity of Computations by Networks , 1987, IBM J. Res. Dev..

[18]  Earl E. Swartzlander,et al.  A Spanning Tree Carry Lookahead Adder , 1992, IEEE Trans. Computers.

[19]  Robert K. Montoye.  AREA-Time Efficient Addition in Charge Based Technology , 1981, 18th Design Automation Conference.

[20]  Mary Jane Irwin,et al.  Regular, area-time efficient carry-lookahead adders , 1985, IEEE Symposium on Computer Arithmetic.

[21]  Joos Vandewalle,et al.  On the Circuit Complexity of Feedforward Neural Networks , 1994 .

[22]  N. P. Red’kin.  Synthesis of threshold circuits for certain classes of Boolean functions , 1970 .

[23]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[24]  Roman Smolensky,et al.  Algebraic methods in the theory of lower bounds for Boolean circuit complexity , 1987, STOC.

[25]  Joos Vandewalle,et al.  Area-Time Performances of Some Neural Computations , 1994 .

[26]  Jehoshua Bruck,et al.  Neural computation of arithmetic functions , 1990 .

[27]  José Fernández Ramos,et al.  Two Operand Binary Adders with Threshold Logic , 1999, IEEE Trans. Computers.

[28]  Theodoros Andronikos,et al.  Modelling the complexity of parallel and VLSI computations with Boolean circuits , 1995, Microprocess. Microsystems.

[29]  Michael E. Saks,et al.  On threshold circuits for parity , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[30]  T. Kailath,et al.  Computing With Almost Optimal Size Threshold Circuits , 1991, Proceedings. 1991 IEEE International Symposium on Information Theory.

[31]  Marc Tremblay,et al.  A 64-b microprocessor with multimedia support , 1995 .

[32]  Robert W. Doran.  Variants of an Improved Carry Look-Ahead Adder , 1988, IEEE Trans. Computers.

[33]  Mary Jane Irwin,et al.  ELM-A Fast Addition Algorithm Discovered by a Program , 1992, IEEE Trans. Computers.

[34]  Valeriu Beiu,et al.  On the circuit and VLSI complexity of threshold gate COMPARISON , 1998, Neurocomputing.

[35]  J. Reif,et al.  On Threshold Circuits and Polynomial Computation , 1992, SIAM J. Comput..

[36]  Tom Rhyne.  Limitations on Carry Lookahead Networks , 1984, IEEE Transactions on Computers.

[37]  Jehoshua Bruck,et al.  Harmonic Analysis of Polynomial Threshold Functions , 1990, SIAM J. Discret. Math..

[38]  Steven P. Levitan,et al.  VLSI DESIGN OF HIGH-SPEED, LOW-AREA ADDITION CIRCUITRY. , 1987 .

[39]  Robert C. Minnick,et al.  Linear-Input Logic , 1961, IRE Trans. Electron. Comput..

[40]  Harold S. Stone,et al.  A Parallel Algorithm for the Efficient Solution of a General Class of Recurrence Equations , 1973, IEEE Transactions on Computers.

[41]  Rudy Lauwereins,et al.  Comparison and Threshold Gate Decomposition , 1993 .

[42]  Kai-Yeung Siu,et al.  On Optimal Depth Threshold Circuits for Multiplication and Related Problems , 1994, SIAM J. Discret. Math..

[43]  Thomas Kailath,et al.  Depth-Size Tradeoffs for Neural Computation , 1991, IEEE Trans. Computers.

[44]  Robert C. Williamson.  e-Entropy and the Complexity of Feedforward Neural Networks , 1990, NIPS 1990.

[45]  Anwar Adkhamovich Irmatov,et al.  On the number of threshold functions , 1993 .

[46]  Johan Håstad,et al.  On the Size of Weights for Threshold Gates , 1994, SIAM J. Discret. Math..

[47]  Johan Håstad,et al.  Almost optimal lower bounds for small depth circuits , 1986, STOC '86.

[48]  Valeriu Beiu.  Neural Addition and Fibonacci Numbers , 1999, IWANN.

[49]  Samuel D. Naffziger,et al.  The implementation of the Itanium 2 microprocessor , 2002, IEEE J. Solid State Circuits.

[50]  Joseph W. Goodman,et al.  On the power of neural networks for solving hard problems , 1990, J. Complex..

[51]  Rudy Lauwereins,et al.  Placing Feedforward Neural Networks Among Several Circuit Complexity Classes , 1994 .

[52]  Huey Ling.  High Speed Binary Adder , 1981, IBM J. Res. Dev..

[53]  Stamatis Vassiliadis,et al.  2-1 Additions and Related Arithmetic Operations with Threshold Logic , 1996, IEEE Trans. Computers.

[54]  John Shawe-Taylor,et al.  Classes of feedforward neural networks and their circuit complexity , 1992, Neural Networks.

[55]  Alon Orlitsky,et al.  On the Circuit Complexity of Neural Networks , 1990, NIPS.

[56]  Georg Schnitger,et al.  On the computational power of sigmoid versus Boolean threshold circuits , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[57]  Kai-Yeung Siu,et al.  Optimal Depth Neural Networks for Multiplication and Related Problems , 1992, NIPS.

[58]  Russell Beale,et al.  Handbook of Neural Computation , 1996 .

[59]  Rudy Lauwereins,et al.  Addition Using Constrained Threshold Gates , 1994 .

[60]  Gary G. R. Green,et al.  Neural networks, approximation theory, and finite precision computation , 1995, Neural Networks.

[61]  Michael J. Flynn,et al.  High-Speed Addition in CMOS , 1992, IEEE Trans. Computers.

[62]  C. Yeh,et al.  Optimal-depth threshold circuits for multiplication and related problems , 1999, Conference Record of the Thirty-Third Asilomar Conference on Signals, Systems, and Computers (Cat. No.CH37020).

[63]  C. Yeh,et al.  Optimal-depth circuits for prefix computation and addition , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[64]  Jehoshua Bruck,et al.  Neural Computing with Small Weights , 1991, NIPS.

[65]  A. Razborov.  Lower bounds on the size of bounded depth circuits over a complete basis with logical addition , 1987 .

[66]  Andrew Chi-Chih Yao,et al.  ON ACC and threshold circuits , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[67]  M. G. Johnson.  A symmetric CMOS NOR gate for high-speed applications , 1988 .

[68]  Noga Alon,et al.  Explicit Constructions of Depth-2 Majority Circuits for Comparison and Addition , 1994, SIAM J. Discret. Math..

[69]  Neil Immerman,et al.  The complexity of iterated multiplication , 1989, [1989] Proceedings. Structure in Complexity Theory Fourth Annual Conference.

[70]  Jehoshua Bruck,et al.  Polynomial threshold functions, AC functions and spectrum norms , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.