Guessing Subject to Distortion

We investigate the problem of guessing a random vector X within distortion level D. Our aim is to characterize the best attainable performance in the sense of minimizing, in some probabilistic sense, the number of required guesses G(X) until the error falls below D. The underlying motivation is that G(X) is the number of candidate codewords to be examined by a rate-distortion block encoder until a satisfactory codeword is found. In particular, for memoryless sources, we provide a single-letter characterization of the least achievable exponential growth rate of the /spl rho/th moment of G(X) as the dimension of the random vector X grows without bound. In this context, we propose an asymptotically optimal guessing scheme that is universal both with respect to the information source and the value of /spl rho/. We then study some properties of the exponent function E(D, /spl rho/) along with its relation to the source-coding exponents. Finally, we provide extensions of our main results to the Gaussian case, guessing with side information, and sources with memory.

[1]  Pierre A. Humblet Generalization of Huffman coding to minimize the probability of buffer overflow , 1981, IEEE Trans. Inf. Theory.

[2]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[3]  A. Rényi On Measures of Entropy and Information , 1961 .

[4]  Richard E. Blahut,et al.  Principles and practice of information theory , 1987 .

[5]  A. Wyner On the Probability of Buffer Overflow Under an Arbitrary Bounded Input-Output Distribution , 1974 .

[6]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[7]  Bin Yu,et al.  A rate of convergence result for a universal D-semifaithful code , 1993, IEEE Trans. Inf. Theory.

[8]  Neri Merhav On list size exponents in rate-distortion coding , 1997, IEEE Trans. Inf. Theory.

[9]  Rudolf Ahlswede,et al.  Extremal properties of rate distortion functions , 1990, IEEE Trans. Inf. Theory.

[10]  R. Gallager Information Theory and Reliable Communication , 1968 .

[11]  J. Ziv,et al.  On the optimal asymptotic performance of universal ordering and of discrimination of individual sequences , 1992, IEEE Trans. Inf. Theory.

[12]  Frederick Jelinek,et al.  Buffer overflow in variable length coding of fixed rate sources , 1968, IEEE Trans. Inf. Theory.

[13]  Neri Merhav Universal decoding for memoryless Gaussian channels with a deterministic interference , 1993, IEEE Trans. Inf. Theory.

[14]  William Equitz,et al.  Successive refinement of information , 1991, IEEE Trans. Inf. Theory.

[15]  Imre Csiszár Generalized cutoff rates and Renyi's information measures , 1995, IEEE Trans. Inf. Theory.

[16]  I. Csiszár Generalized Cutoff Rates and Renyi's Information Measures , 1993, Proceedings. IEEE International Symposium on Information Theory.

[17]  Katalin Marton,et al.  Error exponent for source coding with a fidelity criterion , 1974, IEEE Trans. Inf. Theory.

[18]  Neri Merhav,et al.  Universal coding with minimum probability of codeword length overflow , 1991, IEEE Trans. Inf. Theory.

[19]  Erdal Arikan An inequality on guessing and its application to sequential decoding , 1996, IEEE Trans. Inf. Theory.