Rate-distortion methods for image and video compression

In this article we provide an overview of rate-distortion (R-D) based optimization techniques and their practical application to image and video coding. We begin with a short discussion of classical rate-distortion theory and then we show how in many practical coding scenarios, such as in standards-compliant coding environments, resource allocation can be put in an R-D framework. We then introduce two popular techniques for resource allocation, namely, Lagrangian optimization and dynamic programming. After a discussion of these techniques as well as some of their extensions, we conclude with a quick review of literature in these areas citing a number of applications related to image and video compression and transmission.

[1]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[2]  W. R. Bennett,et al.  Spectra of quantized signals , 1948, Bell Syst. Tech. J..

[3]  Joel Max,et al.  Quantizing for minimum distortion , 1960, IRE Trans. Inf. Theory.

[4]  P. Zador.  DEVELOPMENT AND EVALUATION OF PROCEDURES FOR QUANTIZING MULTIVARIATE DISTRIBUTIONS , 1963 .

[5]  Harvey J. Everett.  Generalized Lagrange Multiplier Method for Solving Problems of Optimum Allocation of Resources , 1963 .

[6]  Thomas M. Cover,et al.  Broadcast channels , 1972, IEEE Trans. Inf. Theory.

[7]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[8]  Allen Gersho,et al.  Asymptotically optimal block quantization , 1979, IEEE Trans. Inf. Theory.

[9]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[10]  David G. Daut,et al.  Combined Source-Channel Coding of Images Using the Block Cosine Transform , 1981, IEEE Trans. Commun..

[11]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[12]  Shu Lin,et al.  Error control coding : fundamentals and applications , 1983 .

[13]  Masao Kasahara,et al.  A construction of vector quantizers for noisy channels , 1984 .

[14]  Nariman Farvardin,et al.  Optimum quantizer performance for a class of non-Gaussian memoryless sources , 1984, IEEE Trans. Inf. Theory.

[15]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[16]  K. Zeger,et al.  Zero redundancy channel coding in vector quantisation , 1987 .

[17]  Jerry D. Gibson,et al.  Digital coding of waveforms: Principles and applications to speech and video , 1985, Proceedings of the IEEE.

[18]  Joachim Hagenauer,et al.  Rate-compatible punctured convolutional codes (RCPC codes) and their applications , 1988, IEEE Trans. Commun..

[19]  Jan Biemond,et al.  An optimal bit allocation algorithm for sub-band coding , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[20]  Yair Shoham,et al.  Efficient bit allocation for an arbitrary set of quantizers [speech coding] , 1988, IEEE Trans. Acoust. Speech Signal Process..

[21]  Philip A. Chou,et al.  Optimal pruning with applications to tree-structured source coding and modeling , 1989, IEEE Trans. Inf. Theory.

[22]  R. Gray.  Source Coding Theory , 1989 .

[23]  Philip A. Chou,et al.  Entropy-constrained vector quantization , 1989, IEEE Trans. Acoust. Speech Signal Process..

[24]  Michael W. Marcellin,et al.  Trellis coded quantization of memoryless and Gauss-Markov sources , 1990, IEEE Trans. Commun..

[25]  Allen Gersho,et al.  Rate-constrained optimal block-adaptive coding for digital tape recording of HDTV , 1991, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[27]  Thomas R. Fischer,et al.  Joint trellis coded quantization/modulation , 1991, IEEE Trans. Commun..

[28]  William F. Schreiber.  Considerations in the design of HDTV systems for terrestrial broadcasting , 1991 .

[29]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[30]  Nariman Farvardin,et al.  On the performance and complexity of channel-optimized vector quantizers , 1991, IEEE Trans. Inf. Theory.

[31]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[32]  Gary J. Sullivan,et al.  Recursive optimal pruning with applications to tree structured vector quantizers , 1992, IEEE Trans. Image Process..

[33]  Antonio Ortega,et al.  Multiresolution broadcast for digital HDTV using joint source-channel coding , 1992, [Conference Record] SUPERCOMM/ICC '92 Discovering a New World of Communications.

[34]  Joan L. Mitchell,et al.  JPEG: Still Image Data Compression Standard , 1992 .

[35]  Min Wang,et al.  Entropy-constrained trellis-coded quantization , 1992, IEEE Trans. Inf. Theory.

[36]  Amy R. Reibman,et al.  Constraints on variable bit-rate video for ATM networks , 1992, IEEE Trans. Circuits Syst. Video Technol..

[37]  Kannan Ramchandran,et al.  Tilings of the time-frequency plane: construction of arbitrary orthogonal bases and fast tiling algorithms , 1993, IEEE Trans. Signal Process..

[38]  Allen Gersho,et al.  Rate-constrained picture-adaptive quantization for JPEG baseline coders , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[39]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[40]  Vladimir Cuperman,et al.  Joint source and channel coding using a non-linear receiver , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[41]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[42]  Robert J. Safranek,et al.  Signal compression based on models of human perception , 1993, Proc. IEEE.

[43]  Reginald L. Lagendijk,et al.  Forward rate control for MPEG recording , 1993, Other Conferences.

[44]  David W. Lin,et al.  Optimal delayed-coding of video sequences subject to a buffer-size constraint , 1993, Other Conferences.

[45]  K Ramchandran,et al.  Best wavelet packet bases in a rate-distortion sense , 1993, IEEE Trans. Image Process..

[46]  Jerome M. Shapiro,et al.  Optimal bit allocation in the presence of quantizer feedback , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[47]  Nariman Farvardin,et al.  A structured fixed-rate vector quantizer derived from a variable-length scalar quantizer - II: Vector sources , 1993, IEEE Trans. Inf. Theory.

[48]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[49]  Cheng-Tie Chen,et al.  A self-governing rate buffer control strategy for pseudoconstant bit rate video coding , 1993, IEEE Trans. Image Process..

[50]  K. Ramchandran,et al.  Flexible time segmentations for time-varying wavelet packets , 1994, Proceedings of IEEE-SP International Symposium on Time- Frequency and Time-Scale Analysis.

[51]  Antonio Ortega,et al.  Bit allocation for dependent quantization with applications to multiresolution and MPEG video coders , 1994, IEEE Trans. Image Process..

[52]  Jinho Choi,et al.  A stable feedback control of the buffer state using the controlled Lagrange multiplier method , 1994, IEEE Trans. Image Process..

[53]  Antonio Ortega,et al.  Optimal trellis-based buffered compression and fast approximations , 1994, IEEE Trans. Image Process..

[54]  Roberto H. Bamberger,et al.  Optimum classification in subband coding of images , 1994, Proceedings of 1st International Conference on Image Processing.

[55]  Kannan Ramchandran,et al.  Rate-distortion optimal fast thresholding with complete JPEG/MPEG decoder compatibility , 1994, IEEE Trans. Image Process..

[56]  Gary J. Sullivan,et al.  Efficient quadtree coding of images and video , 1994, IEEE Trans. Image Process..

[57]  Antonio Ortega,et al.  Rate control for video coding over variable bit rate channels with applications to wireless transmission , 1995, Proceedings., International Conference on Image Processing.

[58]  Jiro Katto,et al.  Mathematical analysis of MPEG compression capability and its application to rate control , 1995, Proceedings., International Conference on Image Processing.

[59]  Jelena Kovacevic,et al.  Wavelets and Subband Coding , 2013, Prentice Hall Signal Processing Series.

[60]  Antonio Ortega,et al.  Forward-adaptive quantization with optimal overhead cost for image and video coding with applications to MPEG video coders , 1995, Electronic Imaging.

[61]  David W. Lin,et al.  Optimal coding of video sequence over ATM networks , 1995, Proceedings., International Conference on Image Processing.

[62]  Wilson C. Chung,et al.  A new approach to scalable video coding , 1995, Proceedings DCC '95 Data Compression Conference.

[63]  R. Schafer.  Terrestrial transmission of DTVB signals-the European specification , 1995 .

[64]  Arun N. Netravali,et al.  Digital Pictures: Representation, Compression and Standards , 1995 .

[65]  David L. Neuhoff,et al.  Optimal bit allocations for lossless video coders: motion vectors vs. difference frames , 1995, Proceedings., International Conference on Image Processing.

[66]  Roger J. Clarke,et al.  Digital compression of still images and video , 1995 .

[67]  Alexandros Eleftheriadis,et al.  Constrained and general dynamic rate shaping of compressed digital video , 1995, Proceedings., International Conference on Image Processing.

[68]  Aggelos K. Katsaggelos,et al.  A video compression scheme with optimal bit allocation between displacement vector field and displaced frame difference , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[69]  Wei Ding,et al.  Rate control of MPEG video coding and recording by rate-quantization modeling , 1996, IEEE Trans. Circuits Syst. Video Technol..

[70]  Antonio Ortega.  Optimal bit allocation under multiple rate constraints , 1996, Proceedings of Data Compression Conference - DCC '96.

[71]  Truong Q. Nguyen,et al.  Wavelets and filter banks , 1996 .

[72]  Eric Dubois,et al.  Low bit-rate video transmission over fading channels for wireless microcellular systems , 1996, IEEE Trans. Circuits Syst. Video Technol..

[73]  Bede Liu,et al.  Joint encoder and channel rate control of VBR video over ATM networks , 1996, Electronic Imaging.

[74]  M. Vetterli,et al.  Wavelets, subband coding, and best bases , 1996, Proc. IEEE.

[75]  Michael T. Orchard,et al.  Space-frequency quantization for wavelet image coding , 1996, Optics & Photonics.

[76]  Antonio Ortega,et al.  A novel hybrid technique for discrete rate-distortion optimization with applications to fast codebook search for SVQ , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[77]  Sanjit K. Mitra,et al.  Rate-distortion optimized mode selection for very low bit rate video coding and the emerging H.263 standard , 1996, IEEE Trans. Circuits Syst. Video Technol..

[78]  Arun N. Netravali,et al.  Digital Video: An introduction to MPEG-2 , 1996 .

[79]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[80]  Antonio Ortega,et al.  Adaptive quantization of image subbands with efficient overhead rate selection , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[81]  Joan L. Mitchell,et al.  MPEG Video Compression Standard , 1996, Springer US.

[82]  Aggelos K. Katsaggelos,et al.  Rate-Distortion Based Video Compression: Optimal Video Frame Compression and Object Boundary Encoding , 1996 .

[83]  David W. Lin,et al.  Optimal bit allocation for video coding under multiple constraints , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[84]  Hsueh-Ming Hang,et al.  Source model for transform video coder and its application. II. Variable frame rate coding , 1997, IEEE Trans. Circuits Syst. Video Technol..

[85]  A. Katsaggelos,et al.  A video compression scheme with optimal bit allocation among segmentation, motion, and residual error , 1997, IEEE Trans. Image Process..

[86]  Michael T. Orchard,et al.  Flexible tree-structured signal expansions using time-varying wavelet packets , 1997, IEEE Trans. Signal Process..

[87]  Michael T. Orchard,et al.  Image coding based on mixture modeling of wavelet coefficients and a fast estimation-quantization framework , 1997, Proceedings DCC '97. Data Compression Conference.

[88]  Steven McCanne,et al.  Low-Complexity Video Coding for Receiver-Driven Layered Multicast , 1997, IEEE J. Sel. Areas Commun..

[89]  Konstantinos Konstantinides,et al.  Image and Video Compression Standards: Algorithms and Architectures , 1997 .

[90]  Antonio Ortega,et al.  Joint Selection of Source and Channel Rate for VBR Video Transmission Under ATM Policing Constraints , 1997, IEEE J. Sel. Areas Commun..

[91]  C. Chrysafis,et al.  Efficient context-based entropy coding for lossy wavelet image compression , 1997, Proceedings DCC '97. Data Compression Conference.

[92]  Antonio Ortega,et al.  Rate control for robust video transmission over wireless channels , 1997, Electronic Imaging.

[93]  Michael W. Marcellin,et al.  Comparison of different methods of classification in subband coding of images , 1997, IEEE Trans. Image Process..

[94]  Aggelos K. Katsaggelos,et al.  The minimum-average and minimum-maximum criteria in lossy compression , 1997 .

[95]  Aggelos K. Katsaggelos,et al.  Optimal bit allocation among dependent quantizers for the minimum maximum distortion criterion , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[96]  David W. Lin,et al.  Optimal Bit Allocation for Coding of Video Signals over ATM Networks , 1997, IEEE J. Sel. Areas Commun..

[97]  Wei Ding.  Joint encoder and channel rate control of VBR video over ATM networks , 1997, IEEE Trans. Circuits Syst. Video Technol..

[98]  Hsueh-Ming Hang,et al.  Source model for transform video coder and its application. I. Fundamental theory , 1997, IEEE Trans. Circuits Syst. Video Technol..

[99]  Aggelos K. Katsaggelos,et al.  Rate-Distortion Based Video Compression , 1997, Springer US.

[100]  Jungwoo Lee,et al.  Rate-distortion optimized frame type selection for MPEG encoding , 1997, IEEE Trans. Circuits Syst. Video Technol..

[101]  Aggelos K. Katsaggelos,et al.  A Theory for the Optimal Bit Allocation Between Displacement Vector Field and Displaced Frame Difference , 1997, IEEE J. Sel. Areas Commun..

[102]  Kannan Ramchandran,et al.  Hybrid compressed-uncompressed framework for wireless image transmission , 1997, Proceedings of International Conference on Image Processing.

[103]  Kannan Ramchandran,et al.  Joint thresholding and quantizer selection for transform image coding: entropy-constrained analysis and applications to baseline JPEG , 1997, IEEE Trans. Image Process..

[104]  Jeffrey Scott Vitter,et al.  Lexicographic Bit Allocation for MPEG Video , 1997, J. Vis. Commun. Image Represent..

[105]  Antonio Ortega,et al.  Bit-rate control using piecewise approximated rate-distortion characteristics , 1998, IEEE Trans. Circuits Syst. Video Technol..

[106]  Philip M. Long,et al.  Efficient cost measures for motion estimation at low bit rates , 1998, IEEE Trans. Circuits Syst. Video Technol..

[107]  Alan N. Willson,et al.  Rate-distortion optimal motion estimation algorithms for motion-compensated transform video coding , 1998, IEEE Trans. Circuits Syst. Video Technol..

[108]  Gary J. Sullivan,et al.  Rate-distortion optimization for video compression , 1998, IEEE Signal Process. Mag..

[109]  Michael G. Strintzis,et al.  Motion and disparity field estimation using rate-distortion optimization , 1998, IEEE Trans. Circuits Syst. Video Technol..

[110]  David L. Neuhoff,et al.  Optimizing block size in motion-compensated video coding , 1998, J. Electronic Imaging.

[111]  Aggelos K. Katsaggelos,et al.  An optimal polygonal boundary encoding scheme in the rate distortion sense , 1998, IEEE Trans. Image Process..

[112]  Aggelos K. Katsaggelos,et al.  An optimal quadtree-based motion estimation and motion-compensated interpolation scheme for video compression , 1998, IEEE Trans. Image Process..

[113]  Kannan Ramchandran,et al.  Multiresolution Joint Source-Channel Coding for Wireless Channels , 1998 .

[114]  Kannan Ramchandran,et al.  Robust image transmission over energy-constrained time-varying channels using multiresolution joint source-channel coding , 1998, IEEE Trans. Signal Process..

[115]  Michael T. Orchard,et al.  Wavelet packet image coding using space-frequency quantization , 1998, IEEE Trans. Image Process..

[116]  Antonio Ortega,et al.  A Lagrangian optimization approach to rate control for delay-constrained video transmission over burst-error channels , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[117]  Antonio Ortega,et al.  Rate control for robust video transmission over burst-error wireless channels , 1999, IEEE J. Sel. Areas Commun..