Learning From Examples and VSLI Implementation of Neural Networks

The paper details a direct design alternative to the learning techniques used for determin ing the synaptic weights of a neural network, optimizing the area of its VLSI implementation. We consider binary neurons having a t hreshold nonlinear transfer function. The problem to be solved is to find a network when m examples of n input bits are given. The optimum criterion is changed from size-and-depth of the network, to the classical AT 2 complexity measure of VLSI circuits ( A is the area of the chip, and T is the time for propagating the inputs to the outputs). Considering the maximum fan-in of one neuron as a parameter we proceed to show its influence on the area, and suggest how to obtain a full class of solutions. Results are promising, and further directions for research are pointed out in the conclusions, together with some open questions.

[1]  Santosh S. Venkatesh,et al.  The Devil and the Network: What Sparsity Implies to Robustness and Memory , 1990, NIPS 1990.

[2]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[3]  Jehoshua Bruck,et al.  On The Power Of Threshold Circuits With Small Weights , 1991, Proceedings. 1991 IEEE International Symposium on Information Theory.

[4]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[5]  Jenq-Neng Hwang,et al.  Finite Precision Error Analysis of Neural Network Hardware Implementations , 1993, IEEE Trans. Computers.

[6]  Lex A. Akers,et al.  Training a Limited-Interconnect, Synthetic Neural IC , 1988, NIPS.

[7]  Dan Hammerstrom,et al.  The Connectivity Analysis of Simple Association - or- How Many Connections Do You Need! , 1988 .

[8]  Robert O. Winder,et al.  Threshold logic , 1971, IEEE Spectrum.

[9]  H. T. Kung,et al.  A Regular Layout for Parallel Adders , 1982, IEEE Transactions on Computers.

[10]  N. P. Red’kin.  Synthesis of threshold circuits for certain classes of Boolean functions , 1970 .

[11]  Ron Meir,et al.  Evolving a learning algorithm for the binary perceptron , 1991 .

[12]  P. Raghavan,et al.  Learning in threshold networks , 1988, COLT '88.

[13]  Shaohua Tan,et al.  Efficient algorithm for the design of multilayer feedforward neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[14]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[15]  Jehoshua Bruck,et al.  Polynomial threshold functions, AC functions and spectrum norms , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[16]  Yaser S. Abu-Mostafa,et al.  Learning from hints in neural networks , 1990, J. Complex..

[17]  Michael E. Saks,et al.  On threshold circuits for parity , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[18]  John E. Hopcroft,et al.  Synthesis of Minimal Threshold Logic Networks , 1965, IEEE Trans. Electron. Comput..

[19]  Joos Vandewalle,et al.  A Geometric Approach to the Structural Synthesis of Multilayer Perceptron Neural Networks , 1990 .

[20]  ERIC B. BAUM,et al.  On learning a union of half spaces , 1990, J. Complex..

[21]  Rudy Lauwereins,et al.  Efficient decomposition of comparison and its applications , 1993, ESANN.

[22]  Joos Vandewalle,et al.  Overview of Some Efficient Threshold Gate Decomposition Algorithms , 1993 .

[23]  Saburo Muroga,et al.  The principle of majority decision logical elements and the complexity of their circuits , 1959, IFIP Congress.

[24]  Thomas Kailath,et al.  Depth-Size Tradeoffs for Neural Computation , 1991, IEEE Trans. Computers.

[25]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[26]  Eric B. Baum,et al.  Neural net algorithms that learn in polynomial time from examples and queries , 1991, IEEE Trans. Neural Networks.

[27]  Lynn Conway,et al.  Introduction to VLSI systems , 1978 .

[28]  Eddy Mayoraz,et al.  On the Power of Networks of Majority Functions , 1991, IWANN.

[29]  Jan Gecsei,et al.  Adaptation Algorithms for Binary Tree Networks , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[30]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[31]  R. Lauwereins,et al.  Using Threshold Gates To Implement Sigmoid Nonlinearity , 1992 .

[32]  Eric B. Baum,et al.  On the capabilities of multilayer perceptrons , 1988, J. Complex..

[33]  U. Ramacher,et al.  A geometrical approach to neural network design , 1989, International 1989 Joint Conference on Neural Networks.

[34]  John Shawe-Taylor,et al.  Classes of feedforward neural networks and their circuit complexity , 1992, Neural Networks.