On the equivalence of two-layered perceptrons with binary neurons

We consider two-layered perceptions consisting of N binary input units, K binary hidden units and one binary output unit, in the limit N >> K > or = 1. We prove that the weights of a regular irreducible network are uniquely determined by its input-output map up to some obvious global symmetries. A network is regular if its K weight vectors from the input layer to the K hidden units are linearly independent. A (single layered) perceptron is said to be irreducible if its output depends on every one of its input units; and a two-layered perceptron is irreducible if the K + 1 perceptrons that constitute such network are irreducible. By global symmetries we mean, for instance, permuting the labels of the hidden units. Hence, two irreducible regular two-layered perceptrons that implement the same Boolean function must have the same number of hidden units, and must be composed of equivalent perceptrons.

[1]  Michael L. Dertouzos,et al.  An Approach to Single-Threshold-Element Synthesis , 1964, IEEE Trans. Electron. Comput..

[2]  Paul C. Kainen,et al.  Functionally Equivalent Feedforward Neural Networks , 1994, Neural Computation.

[3]  Sompolinsky,et al.  Statistical mechanics of learning from examples. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[4]  E. M.,et al.  Statistical Mechanics , 2021, Manual for Theoretical Chemistry.

[5]  Eduardo D. Sontag,et al.  For neural networks, function determines form , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[6]  Eytan Domany,et al.  Learning by Choice of Internal Representations , 1988, Complex Syst..

[7]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[8]  M. Kendall,et al.  Kendall's advanced theory of statistics , 1995 .

[9]  Eytan Domany,et al.  Learning by CHIR without Storing Internal Representations , 1990, Complex Syst..

[10]  D. Saad.  Explicit symmetries and the capacity of multilayer neural networks , 1994 .

[11]  R. Hecht-Nielsen.  ON THE ALGEBRAIC STRUCTURE OF FEEDFORWARD NETWORK WEIGHT SPACES , 1990 .

[12]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[13]  M. Opper,et al.  On the ability of the optimal perceptron to generalise , 1990 .

[14]  Blatt,et al.  Computational capabilities of restricted two-layered perceptrons. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.