On the weight and density bounds of polynomial threshold functions

In this report, we show that all n-variable Boolean function can be represented as polynomial threshold functions (PTF) with at most $0.75 \times 2^n$ non-zero integer coefficients and give an upper bound on the absolute value of these coefficients. To our knowledge this provides the best known bound on both the PTF density (number of monomials) and weight (sum of the coefficient magnitudes) of general Boolean functions. The special case of Bent functions is also analyzed and shown that any n-variable Bent function can be represented with integer coefficients less than $2^n$ while also obeying the aforementioned density bound. Finally, sparse Boolean functions, which are almost constant except for $m << 2^n$ number of variable assignments, are shown to have small weight PTFs with density at most $m+2^{n-1}$.

[1]  Abbott,et al.  Storage capacity of generalized networks. , 1987, Physical review. A, General physics.

[2]  Erhan Öztop An Upper Bound on the Minimum Number of Monomials Required to Separate Dichotomies of {1, 1}n , 2006, Neural Computation.

[3]  Ryan O'Donnell,et al.  Extremal properties of polynomial threshold functions , 2003, 18th IEEE Annual Conference on Computational Complexity, 2003. Proceedings..

[4]  Mari ® GuÈler A model with an intrinsic property of learning higher order correlations , 2001, Neural Networks.

[5]  Rolf Eckmiller,et al.  Structural adaptation of parsimonious higher-order neural classifiers , 1994, Neural Networks.

[6]  C. Papadimitriou Vector Analysis of Threshold Functions , 1995 .

[7]  Marii G,et al.  A Binary-input Supervised Neural Unit That Forms Input Dependent Higher-order Synaptic Correlations , .

[8]  Jehoshua Bruck,et al.  Harmonic Analysis of Polynomial Threshold Functions , 1990, SIAM J. Discret. Math..

[9]  Nathan Linial,et al.  Spectral properties of threshold functions , 1994, Comb..

[10]  Erhan Oztop,et al.  An upper bound on the minimum number of monomials required to separate dichotomies of {-1, 1}n. , 2006, Neural computation.

[11]  Johan Håstad,et al.  On the Size of Weights for Threshold Gates , 1994, SIAM J. Discret. Math..

[12]  Timo Neumann,et al.  BENT FUNCTIONS , 2006 .

[13]  Michael Schmitt,et al.  On the Complexity of Computing and Learning with Multiplicative Neural Networks , 2002, Neural Computation.

[14]  M. Saks Slicing the hypercube , 1993 .

[15]  Colin Giles,et al.  Learning, invariance, and generalization in high-order neural networks. , 1987, Applied optics.

[16]  Erhan Öztop,et al.  Minimal Sign Representation of Boolean Functions: Algorithms and Exact Results for Low Dimensions , 2015, Neural Computation.

[17]  Michael Schmitt On the Capabilities of Higher-Order Neurons: A Radial Basis Function Approach , 2005, Neural Computation.

[18]  Kwangjo Kim,et al.  Semi-bent Functions , 1994, ASIACRYPT.

[19]  Sihem Mesnager,et al.  On constructions of bent functions from involutions , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[20]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[21]  Hans Dobbertin,et al.  Construction of Bent Functions and Balanced Boolean Functions with High Nonlinearity , 1994, FSE.