Data Partitioning: A Video Source-Coding Technique for Layered Video and Error Resilience

Data partitioning is a source-coding technique that has existed in one form or another in the standardized hybrid video codecs up to recent times. In essence, it is a method of prioritizing coding data, resulting in video layers that can be separately communicated across an error-prone network. The Chapter includes the background that has led to data partitioning being included in the standardized codecs. As this Chapter discusses, it differs from scalable video because the output from conventional, single-layer encoders can be converted to multi-layer form, rather than requiring specialist codec extensions. It is shown that the methods of forming the partitions so far employed are: dividing transformed, residual coefficients into two or more layers; and dividing coded data by function into headers, intra-, and inter-coded residuals to form three or more layers. It is also shown how layering naturally combines with protection by channel coding. Used as an error resilience tool, data partitioning presents a low overhead method, suitable for benign as well as bad channels. And in the three-layer variety, error concealment at the decoder can significantly aid the reconstruction of damaged video frames. The Chapter will be of particular interest to developers charged with making a mobile, low-latency, or interactive video streaming application robust, as they can select from the data-partitioning methods and apply them to open-source code of the recent High Efficiency Video Coding (HEVC) codec standard. Broadcast TV can also benefit from data partitioning. Developers of codecs additionally will find in this Chapter a guide to research and ideas about data partitioning which could be incorporated into future codecs. Data Partitioning: A Video Source-Coding Technique for Layered Video and Error Resilience

[1]  Muhammad Altaf,et al.  H.264 video streaming with data-partitioning and Growth codes , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[2]  Rik Van de Walle,et al.  Constrained Inter Prediction: Removing Dependencies Between Different Data Partitions , 2007, ACIVS.

[3]  Itu-T and Iso Iec Jtc Advanced video coding for generic audiovisual services , 2010 .

[4]  Marta Karczewicz,et al.  The SP- and SI-frames design for H.264/AVC , 2003, IEEE Trans. Circuits Syst. Video Technol..

[5]  John F. Arnold,et al.  Layered coding using bitstream decomposition with drift correction , 1997, IEEE Trans. Circuits Syst. Video Technol..

[6]  Weiping Li,et al.  Overview of fine granularity scalability in MPEG-4 video standard , 2001, IEEE Trans. Circuits Syst. Video Technol..

[7]  Martin Reisslein,et al.  Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison , 2011, IEEE Transactions on Broadcasting.

[8]  D. Huffman A Method for the Construction of Minimum-Redundancy Codes , 1952 .

[9]  P.F.M. Smulders,et al.  Exploiting the 60 GHz band for local wireless multimedia access: prospects and future directions , 2002, IEEE Commun. Mag..

[10]  Dimitris Anastassiou,et al.  Minimal error drift in frequency scalability for motion-compensated DCT coding , 1994, IEEE Trans. Circuits Syst. Video Technol..

[11]  Zhenyu Wu,et al.  An Error Concealment Scheme for Entire Frame Losses for H.264/AVC , 2006, 2006 IEEE Sarnoff Symposium.

[12]  Hiroyuki Katata,et al.  Temporal-scalable coding based on image content , 1997, IEEE Trans. Circuits Syst. Video Technol..

[13]  Thomas Stockhammer,et al.  Dynamic adaptive streaming over HTTP --: standards and design principles , 2011, MMSys.

[14]  Charilaos A. Christopoulos,et al.  Region of interest coding in JPEG 2000 , 2002, Signal Process. Image Commun..

[15]  Harald Ernst,et al.  ATM-Based Multimedia Communication via Satellite , 1999, Eur. Trans. Telecommun..

[16]  Gene Cheung,et al.  Error concealment by data partitioning , 1999, Signal Process. Image Commun..

[17]  Uwe Horn,et al.  Video Encryption Based on Data Partitioning and Scalable Coding - A Comparison , 1998, IDMS.

[18]  Zixiang Xiong,et al.  Expanding Window Fountain codes for scalable video multicast , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[19]  Yao Wang,et al.  Error control and concealment for video communication: a review , 1998, Proc. IEEE.

[20]  Wen Gao,et al.  AVS Video Coding Standard , 2010, Intelligent Multimedia Communication.

[21]  Stephan Wenger,et al.  H.264/AVC over IP , 2003, IEEE Trans. Circuits Syst. Video Technol..

[22]  Siegfried Fößel,et al.  Motion JPEG2000 for high quality video systems , 2003, IEEE Trans. Consumer Electron..

[23]  Yao Wang,et al.  Multiple Description Coding for Video Delivery , 2005, Proceedings of the IEEE.

[24]  Eleftheriadis Alexandros,et al.  Multipoint videoconferencing with scalable video coding , 2006 .

[25]  M. Wada,et al.  Reversible variable length codes , 1995, IEEE Trans. Commun..

[26]  Ian H. Witten,et al.  Arithmetic coding for data compression , 1987, CACM.

[27]  D. J. C. Mackay Fountain codes : Capacity approaching codes design and implementation , 2005 .

[28]  Godred Fairhurst,et al.  The Lightweight User Datagram Protocol (UDP-Lite) , 2004, RFC.

[29]  Mohammed Ghanbari,et al.  Packet prioritization for H.264/AVC video with cyclic intra-refresh line , 2013, J. Vis. Commun. Image Represent..

[30]  Ajay Luthra,et al.  The H.264/AVC Advanced Video Coding standard: overview and introduction to the fidelity range extensions , 2004, SPIE Optics + Photonics.

[31]  Gary J. Sullivan,et al.  High efficiency video coding: the next frontier in video compression [Standards in a Nutshell] , 2013, IEEE Signal Processing Magazine.

[32]  King Ngi Ngan,et al.  Cell-loss concealment techniques for layered video codecs in an ATM network , 1994, IEEE Trans. Image Process..

[33]  Robert H. Morelos-Zaragoza,et al.  The Art of Error Correcting Coding: Morelos-Zaragoza/The Art of Error Correcting Coding, Second Edition , 2006 .

[34]  M. Wien,et al.  The Scalable Video Coding Extension of the H.264/AVC Standard [Standards in a Nutshell] , 2008, IEEE Signal Processing Magazine.

[35]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Heiko Schwarz,et al.  MCTF and scalability extension of H.264/AVC and its application to video transmission, storage, and surveillance , 2005, Visual Communications and Image Processing.

[37]  King Ngi Ngan,et al.  Filters for drift reduction in frequency scalable video coding schemes , 1994 .

[38]  David S. Taubman,et al.  High performance scalable image compression with EBCOT. , 2000, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[39]  Filip De Turck,et al.  On the merits of SVC-based HTTP Adaptive Streaming , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).

[40]  M. Budagavi,et al.  Unequal error protection of MPEG-4 compressed video , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[41]  Muhammad Altaf,et al.  H.264 error resilience performance for wireless video conversational services , 2009, MobiMedia.

[42]  Borko Furht,et al.  Comparing MPEG AVC and SVC for adaptive HTTP streaming , 2012, 2012 IEEE International Conference on Consumer Electronics (ICCE).

[43]  Heiko Schwarz,et al.  Context-based adaptive binary arithmetic coding in the H.264/AVC video compression standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[44]  K. Onthriar,et al.  Performance Comparison of Emerging Dirac Video Codec with H.264/AV , 2006, International Conference on Digital Telecommunications (ICDT'06).

[45]  Aggelos K. Katsaggelos,et al.  Error resilient video coding techniques , 2000, IEEE Signal Process. Mag..

[46]  Ahmet M. Kondoz,et al.  Prioritisation of data partitioned MPEG - 4 video over mobile networks , 2001, Eur. Trans. Telecommun..

[47]  Thomas Stockhammer,et al.  Reliable and Efficient Download Delivery with Raptor Codes , 2006 .

[48]  Gunnar Karlsson,et al.  Packet video and its integration into the network architecture , 1989, IEEE J. Sel. Areas Commun..

[49]  Gary J. Sullivan,et al.  Rate-distortion optimization for video compression , 1998, IEEE Signal Process. Mag..

[50]  Mohammed Ghanbari,et al.  Smoothing transcoded MPEG-1 video streams for Internet transmission , 2004 .

[51]  M. Reha Civanlar,et al.  Packet loss resilience of MPEG-2 scalable video coding algorithms , 1996, IEEE Trans. Circuits Syst. Video Technol..

[52]  Geoffrey Morrison,et al.  A spatially layered hierarchical approach to video coding , 1993, Signal Process. Image Commun..

[53]  Yanling Xu,et al.  H.264 video communication based refined error concealment schemes , 2004, IEEE Transactions on Consumer Electronics.

[54]  T. Stockhammer,et al.  Application Layer FEC in IPTV Services , 2008 .

[55]  Mohammed Ghanbari,et al.  The accuracy of PSNR in predicting video quality for different video scenes and frame rates , 2012, Telecommun. Syst..

[56]  Mohammed Ghanbari,et al.  Two-layer coding of video signals for VBR networks , 1989, IEEE J. Sel. Areas Commun..

[57]  Iain E. Richardson,et al.  The H.264 Advanced Video Compression Standard: Richardson/The H.264 Advanced Video Compression Standard , 2010 .

[58]  Miska M. Hannuksela,et al.  System Layer Integration of High Efficiency Video Coding , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[59]  D. Marpe,et al.  Video coding with H.264/AVC: tools, performance, and complexity , 2004, IEEE Circuits and Systems Magazine.

[60]  Chang Wen Chen,et al.  Video Transmission over MIMO-OFDM System: MDC and Space-Time Coding-Based Approaches , 2007, Adv. Multim..

[61]  Ali C. Begen,et al.  An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP , 2011, MMSys.

[62]  Atul Puri,et al.  Spatial-domain resolution-scalable video coding , 1993, Other Conferences.

[63]  Alexandros Eleftheriadis,et al.  Optimal data partitioning of MPEG-2 coded video , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[64]  Heiko Schwarz,et al.  Analysis of Hierarchical B Pictures and MCTF , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[65]  Gary J. Sullivan,et al.  Spatial Scalability Within the H.264/AVC Scalable Video Coding Extension , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[66]  Thomas Stockhammer,et al.  H.264/AVC data partitioning for mobile video communication , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[67]  Jeffrey Scott Vitter,et al.  Design and analysis of dynamic Huffman codes , 1987, JACM.

[68]  Glen G. Langdon,et al.  An Introduction to Arithmetic Coding , 1984, IBM J. Res. Dev..

[69]  Wesley De Neve,et al.  Flexible macroblock ordering in H.264/AVC , 2006, J. Vis. Commun. Image Represent..

[70]  Henrique S. Malvar,et al.  Low-complexity transform and quantization in H.264/AVC , 2003, IEEE Trans. Circuits Syst. Video Technol..

[71]  Nazanin Rahnavard,et al.  Rateless Codes With Unequal Error Protection Property , 2007, IEEE Transactions on Information Theory.

[72]  Moncef Gabbouj,et al.  Rate adaptation for adaptive HTTP streaming , 2011, MMSys.

[73]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[74]  Gary J. Sullivan,et al.  High Efficiency Video Coding (HEVC), Algorithms and Architectures , 2014, Integrated Circuits and Systems.

[75]  Thomas Stockhammer,et al.  Error-Resilient Coding and Decoding Strategies for Video Communication , 2007 .

[76]  Heiko Schwarz,et al.  Performance Analysis of SVC , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[77]  K. Rijkse,et al.  H.263: video coding for low-bit-rate communication , 1996, IEEE Commun. Mag..

[78]  Mohammed Ghanbari,et al.  Protecting H.264/AVC Data-Partitioned Video Streams over Broadband WiMAX , 2012, Adv. Multim..

[79]  Mohammed Ghanbari,et al.  Video streaming over an Ad Hoc network using data partitioning and path diversity , 2013, IEEE Wireless Communications.

[80]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[81]  Wataru Kameyama,et al.  A Motion Compensation Technique for Down-Scaled Pictures in Layered Coding , 1994 .

[82]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[83]  Colin Bailey,et al.  Pause Intensity: A No-Reference Quality Assessment Metric for Video Streaming in TCP Networks , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[84]  Mohammed Ghanbari,et al.  Intra-Refresh Provision for Data-Partitioned H.264 Video Streaming over WiMAX , 2013 .

[85]  Gary J. Sullivan,et al.  Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC) , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[86]  Miska M. Hannuksela,et al.  H.264/AVC in wireless environments , 2003, IEEE Trans. Circuits Syst. Video Technol..

[87]  Sethuraman Panchanathan,et al.  Error resiliency schemes in H.264/AVC standard , 2006, J. Vis. Commun. Image Represent..

[88]  Faouzi Kossentini,et al.  H.263+: video coding at low bit rates , 1998, IEEE Trans. Circuits Syst. Video Technol..

[89]  Henrique S. Malvar,et al.  The LOT: transform coding without blocking effects , 1989, IEEE Trans. Acoust. Speech Signal Process..

[90]  Mohammed Ghanbari,et al.  Prioritized transmission of data partitioned H.264 video with hierarchical QAM , 2005, IEEE Signal Processing Letters.

[91]  Jani Lainema,et al.  Adaptive deblocking filter , 2003, IEEE Trans. Circuits Syst. Video Technol..