Optimization of Hybridized Error Concealment for H.264

Transmission of highly compressed video bitstreams can result in packet erasures when channel status is unfavorable, the consequence being not only the corruption of a single frame, but propagation to its successors. In order to avoid error-catalyzed artifacts from producing visible corruption of affected video frames, the use of error concealment at the video decoder becomes essential. The purpose of this paper proposes an efficient and integrated novel EC method for the latest video compression standard H.264/AVC, using not only spatially and temporally correlated information but also the tandem utilization of two new coding tools: directional spatial prediction for intracoding and variable block size motion compensation of H.264/AVC. Experiments performed using the proposed hybridization method of combining the above spatial and temporal estimation elements fulfilled the expectations of control-whole-scheme. The experimental results show that the proposed method offers excellent gains of up to 10.62 dB compared to that of the Joint Model (JM) decoder for a wide range of benchmark sequences without any considerable increase in time demand.

[1]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[2]  André Kaup,et al.  Spatial error concealment of corrupted image data using frequency selective extrapolation , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[3]  Teresa H. Y. Meng,et al.  Transform coded image reconstruction exploiting interblock correlation , 1995, IEEE Trans. Image Process..

[4]  N. Canagarajah,et al.  Temporal error concealment using motion field interpolation , 1999 .

[5]  Ioannis Pitas,et al.  Vector rational interpolation schemes for erroneous motion field estimation applied to MPEG-2 error concealment , 2004, IEEE Transactions on Multimedia.

[6]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[7]  Nariman Farvardin,et al.  A perceptually motivated three-component image model-Part I: description of the model , 1995, IEEE Trans. Image Process..

[8]  Tien-Ying Kuo,et al.  Error concealment based on overlapping , 2002, IS&T/SPIE Electronic Imaging.

[9]  Michael W. Marcellin,et al.  POCS-based error concealment for packet video using multiframe overlap information , 1998, IEEE Trans. Circuits Syst. Video Technol..

[10]  Joint Video Team Draft ITU-T Recommendation and Final draft international standard of joint video specification , 2003 .

[11]  Michael R. Frater,et al.  A cell-loss concealment technique for MPEG-2 coded video , 2000, IEEE Trans. Circuits Syst. Video Technol..

[12]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[13]  Ziad Al Kachouh,et al.  Fast DCT-based spatial domain interpolation of blocks in images , 2000, IEEE Trans. Image Process..

[14]  S. Gary,et al.  Joint Model Reference Encoding Methods and Decoding Concealment Methods , 2003 .

[15]  Yanling Xu,et al.  H.264 video communication based refined error concealment schemes , 2004, IEEE Transactions on Consumer Electronics.

[16]  W. Kwok,et al.  Multi-directional interpolation for spatial error concealment , 1993 .

[17]  Dong Tian,et al.  Spatial error concealment based on directional decision and intra prediction , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[18]  E. R. Davies Machine vision , 1990 .

[19]  Yao Wang,et al.  Coding and cell-loss recovery in DCT-based packet video , 1993, IEEE Trans. Circuits Syst. Video Technol..

[20]  A novel selective motion vector matching algorithm for error concealment in MPEG-4 video transmission over error-prone channels , 2003, IEEE Trans. Consumer Electron..

[21]  Weisi Lin,et al.  Visual distortion assessment with emphasis on spatially transitional regions , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  JongWon Kim,et al.  DCT coefficients recovery-based error concealment technique and its application to the MPEG-2 bit stream error , 1997, IEEE Trans. Circuits Syst. Video Technol..

[23]  Lap-Pui Chau,et al.  Efficient motion vector recovery algorithm for H.264 based on a polynomial model , 2005, IEEE Transactions on Multimedia.

[24]  Tien-Ying Kuo,et al.  Fast Variable Block Size Motion Estimation for H.264 Using Likelihood and Correlation of Motion Field , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Lap-Pui Chau,et al.  Error-concealment algorithm for H.26L using first-order plane estimation , 2004, IEEE Transactions on Multimedia.

[26]  Gary J. Sullivan,et al.  Rate-constrained coder control and comparison of video coding standards , 2003, IEEE Trans. Circuits Syst. Video Technol..

[27]  Yo-Sung Ho,et al.  Error concealment techniques for digital TV , 2002, IEEE Trans. Broadcast..

[28]  Wenjun Zeng,et al.  Geometric-structure-based error concealment with novel applications in block-based low-bit-rate coding , 1999, IEEE Trans. Circuits Syst. Video Technol..

[29]  Yao Wang,et al.  A comparison of smoothness measures for error concealment in transform coding , 1995 .

[30]  Li Tang Combined and iterative form of spatial and temporal error concealment for video signals , 2006, IEEE Transactions on Broadcasting.

[31]  Zhou Wang,et al.  Best neighborhood matching: an information loss restoration technique for block-based image coding systems , 1998, IEEE Trans. Image Process..

[32]  Wen Gao,et al.  Improved error concealment algorithms based on H.264/AVC non-normative decoder , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[33]  Pamela C. Cosman,et al.  Decision trees for error concealment in video decoding , 2003, IEEE Trans. Multim..

[34]  Yao Wang,et al.  Error control and concealment for video communication: a review , 1998, Proc. IEEE.

[35]  Mei-Juan Chen,et al.  High-Performance Spatial and Temporal Error-Concealment Algorithms for Block-Based Video Coding Techniques , 2005 .

[36]  David G. Messerschmitt,et al.  Resynchronization of motion compensated video affected by ATM cell loss , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[37]  Yo-Sung Ho,et al.  Error concealment based on directional interpolation , 1997 .

[38]  Mei-Juan Chen,et al.  Motion Vector Based Error Concealment Algorithms , 2002, IEEE Pacific Rim Conference on Multimedia.

[39]  Tao Chen Refined Boundary Matching Algorithm for Temporal Error Concealment , 2002 .

[40]  S. Tsekeridou,et al.  An error concealment scheme for MPEG-2 coded video sequences , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[41]  Ivan V. Bajic Adaptive MAP error concealment for dispersively packetized wavelet-coded images , 2006, IEEE Transactions on Image Processing.

[42]  Ioannis Pitas,et al.  MPEG-2 error concealment based on block-matching principles , 2000, IEEE Trans. Circuits Syst. Video Technol..

[43]  Jin-Jang Leou,et al.  Detection and concealment of transmission errors in H.261 images , 1998, IEEE Trans. Circuits Syst. Video Technol..

[44]  Min-Cheol Hong,et al.  Error concealment algorithms for compressed video , 1999, Signal Process. Image Commun..

[45]  S. Aign,et al.  Temporal and spatial error concealment techniques for hierarchical MPEG-2 video codec , 1995, Proceedings IEEE International Conference on Communications ICC '95.

[46]  Liu Da The Error Concealment Feature in the H.26L Test Model , 2003 .

[47]  C.-C. Jay Kuo,et al.  A spatial-domain error concealment method with edge recovery and selective directional interpolation , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[48]  Luigi Atzori,et al.  A spatio-temporal concealment technique using boundary matching algorithm and mesh-based warping (BMA-MBW) , 2001, IEEE Trans. Multim..

[49]  A Leon-Garcia,et al.  Information loss recovery for block-based image coding techniques-a fuzzy logic approach , 1995, IEEE Trans. Image Process..

[50]  Woo-Jin Song,et al.  Error concealment for MPEG-2 video decoders with enhanced coding mode estimation , 2000, IEEE Trans. Consumer Electron..

[51]  Chang-Su Kim,et al.  Multi-hypothesis error concealment algorithm for H.26L video , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[52]  Bede Liu,et al.  Recovery of lost or erroneously received motion vectors , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[53]  Cedric Nishan Canagarajah,et al.  Error concealment using motion field interpolation , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[54]  Lap-Pui Chau,et al.  A motion vector recovery algorithm for digital video using Lagrange interpolation , 2003, IEEE Trans. Broadcast..

[55]  Sung-Jea Ko,et al.  Robust error concealment for visual communications in burst-packet-loss networks , 2003, IEEE Trans. Consumer Electron..

[56]  Aggelos K. Katsaggelos,et al.  Error resilient video coding techniques , 2000, IEEE Signal Process. Mag..

[57]  Shyi-Chyi Cheng,et al.  Efficient adaptive error concealment technique for video decoding system , 2005, IEEE Transactions on Multimedia.

[58]  Cable Ieee Broadcast IEEE transactions on broadcasting , 1963 .

[59]  Cécile Dufour,et al.  An efficient error concealment implementation for MPEG-4 video streams , 2001, IEEE Trans. Consumer Electron..

[60]  Stephan Wenger,et al.  H.264/AVC over IP , 2003, IEEE Trans. Circuits Syst. Video Technol..

[61]  Yao Wang,et al.  Maximally smooth image recovery in transform coding , 1993, IEEE Trans. Commun..

[62]  Qiang Peng,et al.  Block-based temporal error concealment for video packet using motion vector extrapolation , 2002, IEEE 2002 International Conference on Communications, Circuits and Systems and West Sino Expositions.

[63]  Choong-Woong Lee,et al.  Error concealment technique using projection data for block-based image coding , 1994, Other Conferences.

[64]  Huifang Sun,et al.  Concealment of damaged block transform coded images using projections onto convex sets , 1995, IEEE Trans. Image Process..

[65]  Li-Wei Kang,et al.  A hybrid error concealment scheme for MPEG-2 video transmission based on best neighborhood matching algorithm , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[66]  Ness B. Shroff,et al.  Error Concealment in Encoded Video Streams , 1998 .

[67]  David G. Stork,et al.  Pattern Classification , 1973 .

[68]  Zhang Rongfu,et al.  Content-adaptive spatial error concealment for video communication , 2004 .

[69]  Enrico Magli,et al.  Concealment of whole-frame losses for wireless low bit-rate video based on multiframe optical flow estimation , 2005, IEEE Transactions on Multimedia.

[70]  Wenwu Zhu,et al.  Use of second-order derivative-based smoothness measure for error concealment in transform-based codecs , 1995, Other Conferences.

[71]  Michael T. Orchard,et al.  Novel sequential error-concealment techniques using orientation adaptive interpolation , 2001, IEEE Trans. Circuits Syst. Video Technol..

[72]  Chung-Lin Huang,et al.  Hybrid cell loss concealment methods for MPEG-II-based packet video , 1997, Signal Process. Image Commun..

[73]  Jianle Chen,et al.  Modified edge-oriented spatial interpolation for consecutive blocks error concealment , 2005, IEEE International Conference on Image Processing 2005.

[74]  Cedric Nishan Canagarajah,et al.  Enhanced error concealment with mode selection , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[75]  A. Chimienti,et al.  Frame concealment for H.264/AVC decoders , 2005, 2005 Digest of Technical Papers. International Conference on Consumer Electronics, 2005. ICCE..

[76]  Liang-Gee Chen,et al.  Analysis, fast algorithm, and VLSI architecture design for H.264/AVC intra frame coder , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[77]  Yücel Altunbasak,et al.  Multiframe error concealment for MPEG-coded video delivery over error-prone networks , 2002, IEEE Trans. Image Process..