Fixed-slope universal lossy data compression

Corresponding to any lossless codeword length function l, three universal lossy data compression schemes are presented: one is with a fixed rate, another is with a fixed distortion, and a third is with a fixed slope. The former two universal lossy data compression schemes are the generalization of Yang-Kieffer's (see ibid., vol.42, no.1, p.239-45, 1995) results to the general case of any lossless codeword length function l, whereas the third is new. In the case of fixed-slope /spl lambda/>0, our universal lossy data compression scheme works as follows: for any source sequence x/sup n/ of length n, the encoder first searches for a reproduction sequence y/sup n/ of length n which minimizes a cost function n/sup -1/l(y/sup n/)+/spl lambda//spl rho//sub n/(x/sup n/, y/sup n/) over all reproduction sequences of length n, and then encodes x/sup n/ into the binary codeword of length l(y/sup n/) associated with y/sup n/ via the lossless codeword length function l, where /spl rho//sub n/(x/sup n/, y/sup n/) is the distortion per sample between x/sup n/ and y/sup n/. Under some mild assumptions on the lossless codeword length function l, it is shown that when this fixed-slope data compression scheme is applied to encode a stationary, ergodic source, the resulting encoding rate per sample and the distortion per sample converge with probability one to R/sub /spl lambda// and D/sub /spl lambda//, respectively, where (D/sub /spl lambda//, R/sub /spl lambda//) is the point on the rate distortion curve at which the slope of the rate distortion function is -/spl lambda/. This result holds particularly for the arithmetic codeword length function and Lempel-Ziv codeword length function. The main advantage of this fixed-slope universal lossy data compression scheme over the fixed-rate (fixed-distortion) universal lossy data compression scheme lies in the fact that it converts the encoding problem to a search problem through a trellis and then permits one to use some sequential search algorithms to implement it. Simulation results show that this fixed-slope universal lossy data compression scheme, combined with a suitable search algorithm, is promising.

[1]  Zhen Zhang,et al.  An on-line universal lossy data compression algorithm via continuous codebook refinement - Part II. Optimality for phi-mixing source models , 1996, IEEE Trans. Inf. Theory.

[2]  Andrew J. Viterbi,et al.  Principles of Digital Communication and Coding , 1979 .

[3]  Zhen Zhang,et al.  An on-line universal lossy data compression algorithm by continuous codebook refinement , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[4]  En-Hui Yang,et al.  On the Performance of Data Compression Algorithms Based Upon String Matching , 1998, IEEE Trans. Inf. Theory.

[5]  Jacob Ziv,et al.  Distortion-rate theory for individual sequences , 1980, IEEE Trans. Inf. Theory.

[6]  Kingo Kobayashi,et al.  On asymptotic optimality of a sliding window variation of Lempel-Ziv codes , 1993, IEEE Trans. Inf. Theory.

[7]  Jun Muramatsu,et al.  Dual Quantity of the Distortion-Complexity and a Universal Data-Base for Fixed-Rate Data Compression with Distortion (Special Section on Information Theory and Its Applications) , 1996 .

[8]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[9]  B. Rimoldi,et al.  A universal data compression scheme with distortion , 1990 .

[10]  John C. Kieffer,et al.  Extension of source coding theorems for block codes to sliding-block codes , 1980, IEEE Trans. Inf. Theory.

[11]  杨恩辉 THE PROOF OF LEVIN’S CONJECTURE , 1989 .

[12]  Zhen Zhang,et al.  An on-line universal lossy data compression algorithm via continuous codebook refinement - Part I: Basic results , 1996, IEEE Trans. Inf. Theory.

[13]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[14]  J. Muramatsu,et al.  Distortion-complexity and rate-distortion function , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[15]  John C. Kieffer,et al.  Sample converses in source coding theory , 1991, IEEE Trans. Inf. Theory.

[16]  R. Gray Entropy and Information Theory , 1990, Springer New York.

[17]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[18]  Jorma Rissanen,et al.  Complexity of strings in the class of Markov sources , 1986, IEEE Trans. Inf. Theory.

[19]  En-Hui Yang,et al.  Distortion program-size complexity with respect to a fidelity criterion and rate-distortion function , 1993, IEEE Trans. Inf. Theory.

[20]  D. Ornstein,et al.  Universal Almost Sure Data Compression , 1990 .

[21]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[22]  En-Hui Yang,et al.  Simple universal lossy data compression schemes derived from the Lempel-Ziv algorithm , 1996, IEEE Trans. Inf. Theory.

[23]  Yossef Steinberg,et al.  An algorithm for source coding subject to a fidelity criterion, based on string matching , 1993, IEEE Trans. Inf. Theory.

[24]  D. Ornstein,et al.  The Shannon-McMillan-Breiman theorem for a class of amenable groups , 1983 .

[25]  Paul C. Shields,et al.  The ergodic and entropy theorems revisited , 1987, IEEE Trans. Inf. Theory.

[26]  John C. Kieffer Strong converses in source coding relative to a fidelity criterion , 1991, IEEE Trans. Inf. Theory.

[27]  Takeshi Hashimoto A list-type reduced-constraint generalization of the Viterbi algorithm , 1987, IEEE Trans. Inf. Theory.

[28]  E Yang THE PROOF OF LEVIN'S CONJECTURE , 1989 .

[29]  John C. Kieffer,et al.  A survey of the theory of source coding with a fidelity criterion , 1993, IEEE Trans. Inf. Theory.