A Perspective on Unique Information: Directionality, Intuitions, and Secret Key Agreement

Recently, the partial information decomposition emerged as a promising framework for identifying the meaningful components of the information contained in a joint distribution. Its adoption and practical application, however, have been stymied by the lack of a generally-accepted method of quantifying its components. Here, we briefly discuss the bivariate (two-source) partial information decomposition and two implicitly directional interpretations used to intuitively motivate alternative component definitions. Drawing parallels with secret key agreement rates from information-theoretic cryptography, we demonstrate that these intuitions are mutually incompatible and suggest that this underlies the persistence of competing definitions and interpretations. Having highlighted this hitherto unacknowledged issue, we outline several possible solutions.

[1]  Ueli Maurer,et al.  Unconditionally Secure Key Agreement and the Intrinsic Conditional Information , 1999, IEEE Trans. Inf. Theory.

[2]  Eric Chitambar,et al.  The Conditional Common Information in Classical and Quantum Secret Key Distillation , 2018, IEEE Transactions on Information Theory.

[3]  Joseph T. Lizier,et al.  Pointwise Partial Information DecompositionUsing the Specificity and Ambiguity Lattices , 2018, Entropy.

[4]  Eckehard Olbrich,et al.  Shared Information -- New Insights and Problems in Decomposing Information in Complex Systems , 2012, ArXiv.

[5]  Rudolf Ahlswede,et al.  Common randomness in information theory and cryptography - I: Secret sharing , 1993, IEEE Trans. Inf. Theory.

[6]  U. Maurer,et al.  Secret key agreement by public discussion from common information , 1993, IEEE Trans. Inf. Theory.

[7]  Eckehard Olbrich,et al.  On extractable shared information , 2017, Entropy.

[8]  Eckehard Olbrich,et al.  Quantifying unique information , 2013, Entropy.

[9]  Robin A. A. Ince Measuring multivariate redundant information with pointwise common change in surprisal , 2016, Entropy.

[10]  Amin Gohari,et al.  Coding for Positive Rate in the Source Model Key Agreement Problem , 2017, IEEE Transactions on Information Theory.

[11]  James P. Crutchfield,et al.  dit: a Python package for discrete information theory , 2018, J. Open Source Softw..

[12]  Rudolf Ahlswede,et al.  Common Randomness in Information Theory and Cryptography - Part II: CR Capacity , 1998, IEEE Trans. Inf. Theory.