Video Compression - From Concepts to the H.264/AVC Standard

Over the last one and a half decades, digital video compression technologies have become an integral part of the way we create, communicate, and consume visual information. In this paper, techniques for video compression are reviewed, starting from basic concepts. The rate-distortion performance of modern video compression schemes is the result of an interaction between motion representation techniques, intra-picture prediction techniques, waveform coding of differences, and waveform coding of various refreshed regions. The paper starts with an explanation of the basic concepts of video codec design and then explains how these various features have been integrated into international standards, up to and including the most recent such standard, known as H.264/AVC.

[1]  J. D. Gibson,et al.  Random lag selection in multiframe motion compensation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[2]  T. Nakai,et al.  Error resilient video coding by dynamic replacing of reference pictures , 1996, Proceedings of GLOBECOM'96. 1996 IEEE Global Telecommunications Conference.

[3]  Bernd Girod,et al.  The Efficiency of Motion-Compensating Prediction for Hybrid Coding of Video Sequences , 1987, IEEE J. Sel. Areas Commun..

[4]  Bernd Girod,et al.  Distributed Video Coding , 2005, Proceedings of the IEEE.

[5]  Gary J. Sullivan,et al.  Rate-constrained coder control and comparison of video coding standards , 2003, IEEE Trans. Circuits Syst. Video Technol..

[6]  J.D. Gibson,et al.  Multiframe block motion compensated video coding for wireless channels , 1996, Conference Record of The Thirtieth Asilomar Conference on Signals, Systems and Computers.

[7]  Antonio Ortega,et al.  Bit allocation for dependent quantization with applications to multiresolution and MPEG video coders , 1994, IEEE Trans. Image Process..

[8]  Thomas Wiegand,et al.  Long-term memory motion-compensated prediction , 1999, IEEE Trans. Circuits Syst. Video Technol..

[9]  Bernd Girod,et al.  Motion-compensating prediction with fractional-pel accuracy , 1993, IEEE Trans. Commun..

[10]  Madhukar Budagavi,et al.  Error propagation in motion compensated video over wireless channels , 1997, Proceedings of International Conference on Image Processing.

[11]  N. Ahmed,et al.  Discrete Cosine Transform , 2019, IEEE Transactions on Computers.

[12]  J. Wu.  FOURIER TRANSFORM CODING OF IMAGES , 1985 .

[13]  Allen Gersho,et al.  Enhanced video compression with standardized bit stream syntax , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[14]  Itu-T.  Video coding for low bitrate communication , 1996 .

[15]  Jani Lainema,et al.  Adaptive deblocking filter , 2003, IEEE Trans. Circuits Syst. Video Technol..

[16]  Michael T. Orchard,et al.  Overlapped block motion compensation: an estimation-theoretic approach , 1994, IEEE Trans. Image Process..

[17]  H. Enomoto,et al.  Orthogonal Transform Coding System for Television Signals , 1971 .

[18]  R. L. Baker,et al.  Rate-distortion optimized motion compensation for video compression using fixed or variable size blocks , 1991, IEEE Global Telecommunications Conference GLOBECOM '91: Countdown to the New Millennium. Conference Record.

[19]  Jordi Ribas-Corbera,et al.  A generalized hypothetical reference decoder for H.264/AVC , 2003, IEEE Trans. Circuits Syst. Video Technol..

[20]  F. W. Mounts.  A video encoding system with conditional picture-element replenishment , 1969 .

[21]  Bernd Girod,et al.  Feedback-based error control for mobile video transmission , 1999, Proc. IEEE.

[22]  Gary J. Sullivan,et al.  Motion compensation for video compression using control grid interpolation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[23]  Yair Shoham,et al.  Efficient bit allocation for an arbitrary set of quantizers [speech coding] , 1988, IEEE Trans. Acoust. Speech Signal Process..

[24]  Ness B. Shroff,et al.  Error Concealment in Encoded Video , 1999 .

[25]  Sanjit K. Mitra,et al.  Rate-distortion optimized mode selection for very low bit rate video coding and the emerging H.263 standard , 1996, IEEE Trans. Circuits Syst. Video Technol..

[26]  Alan N. Willson,et al.  Design and optimization of a differentially coded variable block size motion compensation system , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[27]  Henrique S. Malvar,et al.  Low-complexity transform and quantization in H.264/AVC , 2003, IEEE Trans. Circuits Syst. Video Technol..

[28]  F. Rocca,et al.  Interframe Redundancy Reduction of Video Signals Generated by Translating Objects , 1977, IEEE Trans. Commun..

[29]  F. Giorda,et al.  Bandwidth Reduction of Video Signals via Shift Vector Transmission , 1975, IEEE Trans. Commun..

[30]  A. Habibi.  Hybrid Coding of Pictorial Data , 1974, IEEE Trans. Commun..

[31]  Anil K. Jain,et al.  Displacement Measurement and Its Application in Interframe Image Coding , 1981, IEEE Trans. Commun..

[32]  Henning Schulzrinne,et al.  RTP: A Transport Protocol for Real-Time Applications , 1996, RFC.

[33]  Gary J. Sullivan,et al.  Rate-distortion optimization for video compression , 1998, IEEE Signal Process. Mag..

[34]  Rui Zhang,et al.  Video coding with optimal inter/intra-mode switching for packet loss resilience , 2000, IEEE Journal on Selected Areas in Communications.

[35]  Bede Liu,et al.  Recovery of lost or erroneously received motion vectors , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[36]  Bernd Girod,et al.  Efficiency analysis of multihypothesis motion-compensated prediction for video coding , 2000, IEEE Trans. Image Process..

[37]  Qin-Fan Zhu,et al.  Joint source coding, transport processing, and error concealment for H.323-based packet video , 1998, Electronic Imaging.

[38]  Stephan Wenger,et al.  H.264/AVC over IP , 2003, IEEE Trans. Circuits Syst. Video Technol..

[39]  Mark Handley,et al.  SIP: Session Initiation Protocol , 1999, RFC.

[40]  Alan N. Willson,et al.  Rate-distortion optimal motion estimation algorithms for motion-compensated transform video coding , 1998, IEEE Trans. Circuits Syst. Video Technol..

[41]  Gary J. Sullivan,et al.  Multi-hypothesis motion compensation for low bit-rate video coding , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[42]  Antonio Ortega,et al.  Optimal trellis-based buffered compression and fast approximations , 1994, IEEE Trans. Image Process..

[43]  Harvey J. Everett.  Generalized Lagrange Multiplier Method for Solving Problems of Optimum Allocation of Resources , 1963 .

[44]  Bernd Girod,et al.  Standard compatible extension of H.263 for robust video transmission in mobile environments , 1997, IEEE Trans. Circuits Syst. Video Technol..

[45]  Miska M. Hannuksela,et al.  The error concealment feature in the H.26L test model , 2002, Proceedings. International Conference on Image Processing.

[46]  R. Ladner.  Entropy-constrained Vector Quantization , 2000 .

[47]  Markus Flierl,et al.  A locally optimal design algorithm for block-based multi-hypothesis motion-compensated prediction , 1998, Proceedings DCC '98 Data Compression Conference (Cat. No.98TB100225).

[48]  Heiko Schwarz,et al.  Context-based adaptive binary arithmetic coding in the H.264/AVC video compression standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[49]  Iso/iec 14496-2 Information Technology — Coding of Audio-visual Objects — Part 2: Visual , .

[50]  Ness B. Shroff,et al.  Error Concealment in Encoded Video Streams , 1998 .

[51]  Thomas Wedi,et al.  Motion- and aliasing-compensated prediction for hybrid video coding , 2003, IEEE Trans. Circuits Syst. Video Technol..

[52]  Miska M. Hannuksela,et al.  H.264/AVC in wireless environments , 2003, IEEE Trans. Circuits Syst. Video Technol..

[53]  Thomas Wiegand,et al.  Error-resilient video transmission using long-term memory motion-compensated prediction , 2000, IEEE Journal on Selected Areas in Communications.

[54]  Marta Karczewicz,et al.  The SP- and SI-frames design for H.264/AVC , 2003, IEEE Trans. Circuits Syst. Video Technol..

[55]  Thrasyvoulos N. Pappas,et al.  Joint block-based video source/channel coding for packet-switched networks , 1998, Electronic Imaging.

[56]  Thomas Wiegand,et al.  Multi-frame motion compensated prediction for video transmission , 2001 .

[57]  Markus Flierl,et al.  Generalized B pictures and the draft H.264/AVC video-compression standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[58]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[59]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[60]  Antonio Ortega,et al.  Rate-distortion methods for image and video compression , 1998, IEEE Signal Process. Mag..