Scalable Kernel-Based Minimum Mean Square Error Estimator for Accelerated Image Error Concealment

Error concealment (EC) is of great importance for block-based video systems, such as digital video broadcasting or video streaming services. In this paper, we propose a novel scalable spatial EC algorithm that aims at obtaining high quality reconstructions with reduced computational burden. The proposed technique exploits the excellent reconstructing abilities of the kernel-based minimum mean square error (K-MMSE) estimator. We propose to decompose this approach into a set of hierarchically stacked layers. The first layer performs the basic reconstruction that the subsequent layers can eventually refine. In addition, we design a layer management mechanism, based on profiles, that dynamically adapts the use of higher layers to the visual complexity of the area being reconstructed. The proposed technique outperforms other state-of-the-art algorithms and produces high quality reconstructions, equivalent to K-MMSE, while requiring around one tenth of its computational time.

[1]  Guangtao Zhai,et al.  Spatial Error Concealment With an Adaptive Linear Predictor , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Iain E. Richardson,et al.  The H.264 Advanced Video Compression Standard: Richardson/The H.264 Advanced Video Compression Standard , 2010 .

[3]  Antonio M. Peinado,et al.  Spatial Error Concealment Based on Edge Visual Clearness for Image/Video Communication , 2013, Circuits Syst. Signal Process..

[4]  Iain E. G. Richardson,et al.  The H.264 Advanced Video Compression Standard , 2010 .

[5]  Weisi Lin,et al.  Bayesian Error Concealment With DCT Pyramid for Images , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Zhang Rongfu,et al.  Content-adaptive spatial error concealment for video communication , 2004 .

[7]  Said Elnoubi,et al.  An adaptive fast and efficient spatial error concealment technique for block-based video coding systems , 2009, 2009 52nd IEEE International Midwest Symposium on Circuits and Systems.

[8]  Ángel M. Gómez,et al.  Combining Media-Specific FEC and Error Concealment for Robust Distributed Speech Recognition Over Loss-Prone Packet Channels , 2006, IEEE Transactions on Multimedia.

[9]  Yan Zhao,et al.  Spatial Error Concealment Using Directional Extrapolation , 2005, Digital Image Computing: Techniques and Applications (DICTA'05).

[10]  Shih-Chang Hsia,et al.  Fast-efficient shape error concealment technique based on block classification , 2016, IET Image Process..

[11]  Russell M. Mersereau,et al.  The use of Hough transforms in spatial error concealment , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[12]  Hamid R. Rabiee,et al.  Multi-directional spatial error concealment using adaptive edge thresholding , 2012, IEEE Transactions on Consumer Electronics.

[13]  Ching-Lung Ho,et al.  Temporal and spatial error concealment using cooperative game , 2014, 2014 International Conference on Information Science, Electronics and Electrical Engineering.

[14]  Dinh Trieu Duong,et al.  Hybrid Temporal Error Concealment Methods for Block-Based Compressed Video Transmission , 2008, IEEE Transactions on Broadcasting.

[15]  Huifang Sun,et al.  Concealment of damaged block transform coded images using projections onto convex sets , 1995, IEEE Trans. Image Process..

[16]  F. Bossen,et al.  Common test conditions and software reference configurations , 2010 .

[17]  Shao-Yi Chien,et al.  Efficient Spatial-Temporal Error Concealment Algorithm and Hardware Architecture Design for H.264/AVC , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[19]  C.-C. Jay Kuo,et al.  Spatial and Temporal Error Concealment Techniques for Video Transmission Over Noisy Channels , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  André Kaup,et al.  Fast orthogonality deficiency compensation for improved frequency selective image extrapolation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Michael T. Orchard,et al.  Novel sequential error-concealment techniques using orientation adaptive interpolation , 2001, IEEE Trans. Circuits Syst. Video Technol..

[23]  André Kaup,et al.  Complex-Valued Frequency Selective Extrapolation for Fast Image and Video Signal Extrapolation , 2010, IEEE Signal Processing Letters.

[24]  Jianfei Cai,et al.  Image error-concealment via Block-based Bilateral Filtering , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[25]  Antonio M. Peinado,et al.  Kernel-Based MMSE Multimedia Signal Reconstruction and Its Application to Spatial Error Concealment , 2014, IEEE Transactions on Multimedia.

[26]  Søren Holdt Jensen,et al.  Sequential Error Concealment for Video/Images by Sparse Linear Prediction , 2013, IEEE Transactions on Multimedia.

[27]  Thiow Keng Tan,et al.  Overview of HEVC High-Level Syntax and Reference Picture Management , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  King Ngi Ngan,et al.  Full-Reference Video Quality Assessment by Decoupling Detail Losses and Additive Impairments , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  André Kaup,et al.  Inter-Sequence Error Concealment Techniques for Multi-Broadcast TV Reception , 2011, IEEE Transactions on Broadcasting.

[30]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[31]  André Kaup,et al.  Fading Techniques for Error Concealment in Block-Based Video Decoding Systems , 2007, IEEE Transactions on Broadcasting.

[32]  Ness B. Shroff,et al.  Error concealment techniques for encoded video streams , 1995, Proceedings., International Conference on Image Processing.

[33]  Nicola Asuni,et al.  TESTIMAGES: a Large-scale Archive for Testing Visual Devices and Basic Image Processing Algorithms , 2014, STAG.

[34]  Yo-Sung Ho,et al.  Error concealment techniques for digital TV , 2002, IEEE Trans. Broadcast..

[35]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[36]  Hamid Gharavi,et al.  Spatial interpolation algorithm for error concealment , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  Rabab Kreidieh Ward,et al.  An adaptive Markov random field based error concealment method for video communication in an error prone environment , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).