Information theory and the complexity of boolean functions

AbstractThis paper explores the connections between two areas pioneered by Shannon: the transmission of information with a fidelity criterion, and the realization of Boolean functions by networks and formulae. We study three phenomena:1.The effect of the relative number of O's and l's in a function's table on its complexity.2.The effect of the number of unspecified entries in a partially specified function's table on its complexity.3.The effect of the number of errors allowed in the realization of a function on its complexity. Our main result is a precise version of the following statement:The complexity of approximately realizing a partially specified Boolean function, in whose table a fractiond of the entries are unspecified and a fractionp of the specified entries are l'swith errors allowed in a fraction not more thane of the specified entries, is less by the factor (1 −d) [H(p) − H(e)] (whereH(z) = −z log2z −(1 −z) log2 (1 −z) is the binary entropy function) than the complexity of exactly realizing an arbitrary fully specified Boolean function.We also give an intuitively appealing information-theoretic interpretation of the result.

[1]  Arnold Schönhage,et al.  Schnelle Multiplikation großer Zahlen , 1971, Computing.

[2]  S. K Stein,et al.  Two Combinatorial Covering Theorems , 1974, J. Comb. Theory, Ser. A.

[3]  Claude E. Shannon,et al.  The synthesis of two-terminal switching circuits , 1949, Bell Syst. Tech. J..

[4]  Michael J. Flynn,et al.  Logical Network Cost and Entropy , 1973, IEEE Transactions on Computers.

[5]  David E. Muller,et al.  Complexity in Electronic Switching Circuits , 1956, IRE Trans. Electron. Comput..

[6]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[7]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[8]  W. W. Peterson,et al.  Error-Correcting Codes. , 1962 .

[9]  Judea Pearl,et al.  On the Complexity of Inexact Computations , 1975, Inf. Process. Lett..

[10]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[11]  Robert W. Cook,et al.  Design of a Self-Checking Microprogram Control , 1973, IEEE Transactions on Computers.

[12]  Nicholas Pippenger.  Information theory and the complexity of switching networks , 1975, 16th Annual Symposium on Foundations of Computer Science (sfcs 1975).