Guessing Revisited: A Large Deviations Approach

The problem of guessing a random string is revisited. A close relation between guessing and compression is first established. Then it is shown that if the sequence of distributions of the information spectrum satisfies the large deviation property with a certain rate function, then the limiting guessing exponent exists and is a scalar multiple of the Legendre-Fenchel dual of the rate function. Other sufficient conditions related to certain continuity properties of the information spectrum are briefly discussed. This approach highlights the importance of the information spectrum in determining the limiting guessing exponent. All known prior results are then re-derived as example applications of our unifying approach.

[1]  Rajesh Sundaresan,et al.  The Shannon Cipher System With a Guessing Wiretapper: General Sources , 2011, IEEE Transactions on Information Theory.

[2]  J. Massey.  Guessing and entropy , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[3]  Neri Merhav,et al.  Universal coding with minimum probability of codeword length overflow , 1991, IEEE Trans. Inf. Theory.

[4]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[5]  Hiroki Koga,et al.  Information-Spectrum Methods in Information Theory , 2002 .

[6]  S. Varadhan,et al.  Large deviations , 2019, Graduate Studies in Mathematics.

[7]  E. Seneta.  Non-negative matrices;: An introduction to theory and applications , 1973 .

[8]  S. Natarajan,et al.  Large deviations, hypotheses testing, and source coding for finite Markov chains , 1985, IEEE Trans. Inf. Theory.

[9]  Neri Merhav,et al.  The Shannon cipher system with a guessing wiretapper , 1999, IEEE Trans. Inf. Theory.

[10]  Rajesh Sundaresan,et al.  DRDO – IISc Programme on Advanced Research in Mathematical Engineering Guessing Based On Length Functions ( TR-PME-2007-02 ) by , 2007 .

[11]  K. R. Parthasarathy,et al.  Coding Theorems of Classical and Quantum Information Theory , 2013 .

[12]  László L. Stachó,et al.  A note on Ky Fan's minimax theorem , 1982 .

[13]  Richard S. Ellis.  An overview of the theory of large deviations and applications to statistical mechanics. , 1996 .

[14]  Rajesh Sundaresan,et al.  Guessing Under Source Uncertainty , 2006, IEEE Transactions on Information Theory.

[15]  R. Ellis,et al.  Entropy, large deviations, and statistical mechanics , 1985 .

[16]  Aaron D. Wyner,et al.  An Upper Bound on the Entropy Series , 1972, Inf. Control..

[17]  Erdal Arikan.  An inequality on guessing and its application to sequential decoding , 1996, IEEE Trans. Inf. Theory.

[18]  Paul C. Shields,et al.  The positive-divergence and blowing-up properties , 1994 .

[19]  C. E. Pfister,et al.  Renyi entropy, guesswork moments, and large deviations , 2004, IEEE Transactions on Information Theory.

[20]  J. Ziv,et al.  On the optimal asymptotic performance of universal ordering and of discrimination of individual sequences , 1992, IEEE Trans. Inf. Theory.

[21]  David Malone,et al.  Guesswork and entropy , 2004, IEEE Transactions on Information Theory.

[22]  Srinivasa R. S. Varadhan,et al.  Asymptotic probabilities and differential equations , 1966 .

[23]  A. Si,et al.  Entropy,Large Deviations,and Statistical Mechanics , 2011 .

[24]  E. Olivieri,et al.  Large deviations and metastability: Large deviations and statistical mechanics , 2005 .

[25]  R. Ellis,et al.  Large deviations and statistical mechanics , 1985 .

[26]  Neri Merhav,et al.  Guessing Subject to Distortion , 1998, IEEE Trans. Inf. Theory.

[27]  L. L. Campbell,et al.  A Coding Theorem and Rényi's Entropy , 1965, Inf. Control..

[28]  A. Dembo,et al.  Large Deviation Techniques and Applications. , 1994 .

[29]  Paul C. Shields,et al.  The Interactions Between Ergodic Theory and Information Theory , 1998, IEEE Trans. Inf. Theory.