An On-Line Universal Lossy Data Compression Algorithm via Continuous Codebook Refinement - Part III: Redundancy Analysis

For pt.II see ibid., vol.42, p.822-36 (1996). The Gold-washing data compression algorithm is an adaptive vector quantization algorithm with vector dimension n. In this paper, a redundancy problem of the Gold-washing data compression algorithm is considered. It is demonstrated that for any memoryless source with finite alphabet A and generic distribution p and for any R>0, the redundancy of the Gold-washing data compression algorithm with dimension n (defined as the difference between the average performance of the algorithm and the distortion-rate function D(p,R) of p) is upper-bounded by |/sub /spl delta/R///sup /spl delta//D(p,R)|((|A|+2/spl xi/+4 log n)/2n)+/spl sigma/(logn/n) where /sub /spl delta/R///sup /spl delta//D(p,R) is the partial derivative of D(p,R) with respect to R, |A| is the cardinality of A, and /spl xi/>0 is a parameter used to control the threshold in the Gold-washing algorithm. In connection with the results of Zhang, Yang, and Wei (see ibid., vol.43, no.1, p.71-91, 1997) on the redundancy of lossy source coding, this shows that the Gold-washing algorithm has the optimal convergence rate among all adaptive finite-state vector quantizers.

[1]  R. J. Pilc.  The transmission distortion of a source as a function of the encoding block length , 1968 .

[2]  En-Hui Yang,et al.  Sequential codes, lossless compression of individual sequences, and Kolmogorov complexity , 1996, IEEE Trans. Inf. Theory.

[3]  Zhen Zhang,et al.  An on-line universal lossy data compression algorithm via continuous codebook refinement - Part II. Optimality for phi-mixing source models , 1996, IEEE Trans. Inf. Theory.

[4]  En-Hui Yang,et al.  On the Performance of Data Compression Algorithms Based Upon String Matching , 1998, IEEE Trans. Inf. Theory.

[5]  Neri Merhav.  A comment on 'A rate of convergence result for a universal D-semifaithful code' , 1995, IEEE Trans. Inf. Theory.

[6]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[7]  Tamás Linder,et al.  Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding , 1994, IEEE Trans. Inf. Theory.

[8]  Raphail E. Krichevsky,et al.  The performance of universal encoding , 1981, IEEE Trans. Inf. Theory.

[9]  Bin Yu,et al.  A rate of convergence result for a universal D-semifaithful code , 1993, IEEE Trans. Inf. Theory.

[10]  John C. Kieffer,et al.  A survey of the theory of source coding with a fidelity criterion , 1993, IEEE Trans. Inf. Theory.

[11]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[12]  Frederick Jelinek.  Tree encoding of memoryless time-discrete sources with a fidelity criterion , 1969, IEEE Trans. Inf. Theory.

[13]  Wojciech Szpankowski,et al.  A suboptimal lossy data compression based on approximate pattern matching , 1997, IEEE Trans. Inf. Theory.

[14]  Guy Louchard,et al.  On the average redundancy rate of the Lempel-Ziv code , 1997, IEEE Trans. Inf. Theory.

[15]  Zhen Zhang,et al.  On the Redundancy of Lossy Source Coding with Abstract Alphabets , 1999, IEEE Trans. Inf. Theory.

[16]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[17]  Marcelo J. Weinberger,et al.  Upper bounds on the probability of sequences emitted by finite-state sources and on the redundancy of the Lempel-Ziv algorithm , 1992, IEEE Trans. Inf. Theory.

[18]  Aaron D. Wyner,et al.  Improved redundancy of a version of the Lempel-Ziv algorithm , 1995, IEEE Trans. Inf. Theory.

[19]  Andrew J. Viterbi,et al.  Trellis Encoding of memoryless discrete-time sources with a fidelity criterion , 1974, IEEE Trans. Inf. Theory.

[20]  P. Gács,et al.  Bounds on conditional probabilities with applications in multi-user communication , 1976 .

[21]  Zhen Zhang,et al.  The redundancy of source coding with a fidelity criterion: 1. Known statistics , 1997, IEEE Trans. Inf. Theory.

[22]  Aaron D. Wyner,et al.  Fixed data base version of the Lempel-Ziv data compression algorithm , 1991, IEEE Trans. Inf. Theory.

[23]  Zhen Zhang,et al.  An on-line universal lossy data compression algorithm via continuous codebook refinement - Part I: Basic results , 1996, IEEE Trans. Inf. Theory.

[24]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[25]  En-Hui Yang,et al.  On the redundancy of the fixed-database Lempel-Ziv algorithm for phi -mixing sources , 1997, IEEE Trans. Inf. Theory.

[26]  Lee D. Davisson,et al.  Universal noiseless coding , 1973, IEEE Trans. Inf. Theory.

[27]  A. Wyner.  Communication of analog data from a Gaussian source over a noisy channel , 1968 .

[28]  Serap A. Savari,et al.  Redundancy of the Lempel-Ziv incremental parsing rule , 1997, IEEE Trans. Inf. Theory.

[29]  Jorma Rissanen,et al.  Complexity of strings in the class of Markov sources , 1986, IEEE Trans. Inf. Theory.

[30]  Tamás Linder,et al.  On the cost of finite block length in quantizing unbounded memoryless sources , 1996, IEEE Trans. Inf. Theory.

[31]  Aaron D. Wyner,et al.  On the Transmission of Correlated Gaussian Data over a Noisy Channel with Finite Encoding Block Length , 1972, Inf. Control..

[32]  Robert M. Gray,et al.  Time-invariant trellis encoding of ergodic discrete-time sources with a fidelity criterion , 1977, IEEE Trans. Inf. Theory.

[33]  Robert G. Gallager.  Tree encoding for symmetric sources with a distortion measure , 1974, IEEE Trans. Inf. Theory.

[34]  John C. Kieffer,et al.  Extension of source coding theorems for block codes to sliding-block codes , 1980, IEEE Trans. Inf. Theory.

[35]  Tamás Linder,et al.  Fixed-rate universal lossy source coding and rates of convergence for memoryless sources , 1995, IEEE Trans. Inf. Theory.