The communication complexity of correlation

Let X and Y be finite nonempty sets and (X,Y) a pair of random variables taking values in X?Y. We consider communication protocols between two parties, Alice and Bob, for generating X and Y. Alice is provided an x ? X generated according to the distribution of X , and is required to send a message to Bob in order to enable him to generate y ? Y, whose distribution is the same as that of Y|X=x. Both parties have access to a shared random string generated in advance. Let T[X:Y] be the minimum (over all protocols) of the expected number of bits Alice needs to transmit to achieve this. We show that I[X:Y] ? T[X:Y] ? I [X:Y] + 2 log2 (I[X:Y]+ O(1). We also consider the worst case communication required for this problem, where we seek to minimize the average number of bits Alice must transmit for the worst case x ? X. We show that the communication required in this case is related to the capacity C(E) of the channel E, derived from (X,Y) , that maps x ? X to the distribution of Y|X=x. We also show that the required communication T(E) satisfies C(E) ? T(E) ? C (E) + 2 log2 (C(E)+1) + O(1). Using the first result, we derive a direct-sum theorem in communication complexity that substantially improves the previous such result shown by Jain, Radhakrishnan, and Sen [In Proc. 30th International Colloquium of Automata, Languages and Programming (ICALP), ser. Lecture Notes in Computer Science, vol. 2719. 2003, pp. 300-315]. These results are obtained by employing a rejection sampling procedure that relates the relative entropy between two distributions to the communication complexity of generating one distribution from the other.

[1]  Peter W. Shor,et al.  Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem , 2001, IEEE Trans. Inf. Theory.

[2]  Eyal Kushilevitz,et al.  Communication Complexity , 1997, Adv. Comput..

[3]  Ziv Bar-Yossef,et al.  An information statistics approach to data stream and communication complexity , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[4]  Paul W. Cuff,et al.  Communication requirements for generating correlated random variables , 2008, 2008 IEEE International Symposium on Information Theory.

[5]  Noga Alon,et al.  The Probabilistic Method, Second Edition , 2004 .

[6]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[7]  L. H. Harper.  Optimal numberings and isoperimetric problems on graphs , 1966 .

[8]  Jaikumar Radhakrishnan,et al.  The Communication Complexity of Correlation , 2007, IEEE Transactions on Information Theory.

[9]  Andrew Chi-Chih Yao,et al.  Informational complexity and the direct sum problem for simultaneous message complexity , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[10]  A. Winter.  Compression of sources of probability distributions and density operators , 2002, quant-ph/0208131.

[11]  Jaikumar Radhakrishnan,et al.  A property of quantum relative entropy with an application to privacy in quantum communication , 2009, JACM.

[12]  Andrew Chi-Chih Yao,et al.  Probabilistic computations: Toward a unified measure of complexity , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[13]  Aaron D. Wyner,et al.  The common information of two dependent random variables , 1975, IEEE Trans. Inf. Theory.

[14]  B. Bollobás.  Combinatorics: Set Systems, Hypergraphs, Families of Vectors and Combinatorial Probability , 1986 .

[15]  Jaikumar Radhakrishnan,et al.  Prior entanglement, message compression and privacy in quantum communication , 2005, 20th Annual IEEE Conference on Computational Complexity (CCC'05).

[16]  Rahul Jain,et al.  Communication complexity of remote state preparation with entanglement , 2005, Quantum Inf. Comput..

[17]  Jaikumar Radhakrishnan,et al.  A Direct Sum Theorem in Communication Complexity via Message Compression , 2003, ICALP.

[18]  Amit Chakrabarti,et al.  An optimal randomised cell probe lower bound for approximate nearest neighbour searching , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[19]  R. Gallager.  Information Theory and Reliable Communication , 1968 .

[20]  Jaikumar Radhakrishnan,et al.  A lower bound for the bounded round quantum communication complexity of set disjointness , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..