Neural Networks and Soft Computing

This paper discusses size-optimal solutions for implementing arbitrary Boolean functions using threshold gates. After presenting the state-of-the-art, we start from the result of Horne and Hush [12], which shows that threshold gate circuits restricted to fan-in 2 can implement arbitrary Boolean functions, but require O(2/n) gates in 2n layers. This result will be generalized to arbitrary fan-ins (∆), lowering the depth to n/log∆ + n/∆, and proving that all the (relative) minimums of size are obtained for sub-linear fan-ins (∆ < n − logn). The fact that size-optimal solutions have sub-linear fan-ins is encouraging, as the area and the delay of VLSI implementations are related to the fan-in of the gates.

[1]  Don R. Hush,et al.  On the node complexity of neural networks , 1994, Neural Networks.

[2]  Michael I. Jordan,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[3]  Claude E. Shannon,et al.  The synthesis of two-terminal switching circuits , 1949, Bell Syst. Tech. J..

[4]  Valeriu Beiu,et al.  Digital integrated circuit implementations , 1997 .

[5]  Valeriu Beiu,et al.  On the circuit and VLSI complexity of threshold gate COMPARISON , 1998, Neurocomputing.

[6]  Dan Hammerstrom,et al.  The Connectivity Analysis of Simple Association - or- How Many Connections Do You Need! , 1988 .

[7]  Robert C. Minnick,et al.  Linear-Input Logic , 1961, IRE Trans. Electron. Comput..

[8]  Ian Parberry,et al.  Circuit complexity and neural networks , 1994 .

[9]  Joseph W. Goodman,et al.  On the power of neural networks for solving hard problems , 1990, J. Complex..

[10]  M.H. Hassoun,et al.  Fundamentals of Artificial Neural Networks , 1996, Proceedings of the IEEE.

[11]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[12]  Thomas Kailath,et al.  Depth-Size Tradeoffs for Neural Computation , 1991, IEEE Trans. Computers.

[13]  Masahiko Arai,et al.  Bounds on the number of hidden units in binary-valued three-layer neural networks , 1993, Neural Networks.

[14]  Eric B. Baum,et al.  On the capabilities of multilayer perceptrons , 1988, J. Complex..

[15]  Y. F. Huang,et al.  Bounds on number of hidden neurons of multilayer perceptrons in classification and recognition , 1990, IEEE International Symposium on Circuits and Systems.

[16]  Valeriu Beiu,et al.  Tight Bounds on the Size of Neural Networks for Classification Problems , 1997, IWANN.

[17]  Gary G. R. Green,et al.  Neural networks, approximation theory, and finite precision computation , 1995, Neural Networks.