Switchable video error concealment using encoder driven scene transition detection and edge preserving SEC

Error concealment, a decoder based technique, attempts to reconstruct the corrupted regions of a video frame, using spatial and/or temporal correlation. For sequences with high temporal correlations, the performance of Temporal Error Concealment (TEC) techniques are better than Spatial Error Concealment (SEC) techniques in terms of PSNR of the reconstructed frames. However, the performance of TECs deteriorate drastically at the scene boundary due to low temporal correlation with the reference frame. In this paper, we propose a novel encoder controlled transition detection scheme which would facilitate selection of concealment strategy, that makes a choice between our newly proposed edge-direction based SEC technique and a TEC scheme. Exploiting Motion Vector (MV), Sum of Absolute Difference (SAD), and Motion Compensated Histogram Difference (MCHD), scene transition is detected at the encoder. A single bit transition status flag per frame is then sent to the decoder as a part of the header information to select the concealment policy between TEC and SEC. The present work also proposes a new edge-directed spatial error concealment, termed as “Directional Edge Based Spatial Error Concealment” (DEBSEC), which outperforms existing techniques in terms of PSNR and SSIM of the concealed frame.

[1]  Chi-Chun Lo,et al.  Video segmentation using a histogram-based fuzzy c-means clustering algorithm , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[2]  Oscar C. Au,et al.  Edge-Directed Error Concealment , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Soo-Chang Pei,et al.  Novel error concealment method with adaptive prediction to the abrupt and gradual scene changes , 2004, IEEE Trans. Multim..

[4]  Sungjoo Yoo,et al.  Multi-histogram based scene change detection for frame rate up-conversion , 2013, 2013 IEEE International Conference on Consumer Electronics (ICCE).

[5]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying scene breaks , 1995, MULTIMEDIA '95.

[6]  Lei Zhu,et al.  An integrated method for video shot boundary detection , 2010, Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon).

[7]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[8]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[9]  Morshed U. Chowdhury,et al.  Fast Scene Change Detection Based Histogram , 2007, 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007).

[10]  Min Gyo Chung,et al.  A scene boundary detection method , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Patrick Le Callet,et al.  An image quality assessment method based on perception of structural information , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[13]  C.-C. Jay Kuo,et al.  Spatial error concealment with sequence-aligned texture modeling and adaptive directional recovery , 2011, J. Vis. Commun. Image Represent..

[14]  Yap-Peng Tan,et al.  Modified Kolmogorov-Smirnov metric for shot boundary detection , 2003 .

[15]  Cécile Dufour,et al.  An efficient error concealment implementation for MPEG-4 video streams , 2001, IEEE Trans. Consumer Electron..

[16]  Akio Nagasaka,et al.  Automatic Video Indexing and Full-Video Search for Object Appearances , 1991, VDB.

[17]  Oscar C. Au,et al.  Video Error Concealment Using Spatio-Temporal Boundary Matching and Partial Differential Equation , 2008, IEEE Transactions on Multimedia.

[18]  Wenjun Zeng,et al.  Geometric-structure-based error concealment with novel applications in block-based low-bit-rate coding , 1999, IEEE Trans. Circuits Syst. Video Technol..

[19]  Hamid Gharavi,et al.  Spatial interpolation algorithm for error concealment , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Patrick Bouthemy,et al.  A unified approach to shot change detection and camera motion characterization , 1999, IEEE Trans. Circuits Syst. Video Technol..

[21]  Yao Wang,et al.  Maximally smooth image recovery in transform coding , 1993, IEEE Trans. Commun..

[22]  S. Abe,et al.  Content oriented visual interface using video icons for visual database systems , 1989, [Proceedings] 1989 IEEE Workshop on Visual Languages.

[23]  Zhenzhong Chen,et al.  Video Quality Assessment Based on Measuring Perceptual Noise From Spatial and Temporal Perspectives , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Chen Yaowu,et al.  A Novel Scene Change Detection Algorithm for H.264/AVC Bitstreams , 2008, 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application.

[25]  Mohammed Ghanbari,et al.  Intra-frame loss concealment based on directional extrapolation , 2011, Signal Process. Image Commun..

[26]  Y.-N. Li,et al.  Fast video shot boundary detection framework employing pre-processing techniques , 2009, IET Image Process..

[27]  Yao Wang,et al.  Error control and concealment for video communication: a review , 1998, Proc. IEEE.

[28]  Alan Hanjalic,et al.  Shot-boundary detection: unraveled and resolved? , 2002, IEEE Trans. Circuits Syst. Video Technol..

[29]  Bo Yan,et al.  A Hybrid Frame Concealment Algorithm for H.264/AVC , 2010, IEEE Transactions on Image Processing.

[30]  Chong-Wah Ngo,et al.  Video partitioning by temporal slice coherency , 2001, IEEE Trans. Circuits Syst. Video Technol..

[31]  Soo-Chang Pei,et al.  Efficient MPEG Compressed Video Analysis Using Macroblock Type Information , 1999, IEEE Trans. Multim..

[32]  Arnaldo de Albuquerque Araújo,et al.  Video segmentation based on 2D image analysis , 2003, Pattern Recognit. Lett..

[33]  Hamid R. Rabiee,et al.  Multi-directional spatial error concealment using adaptive edge thresholding , 2012, IEEE Transactions on Consumer Electronics.

[34]  Markus Rupp,et al.  SCENE CHANGE DETECTION FOR H.264 USING DYNAMIC THRESHOLD TECHNIQUES , 2005 .

[35]  Miska M. Hannuksela,et al.  The error concealment feature in the H.26L test model , 2002, Proceedings. International Conference on Image Processing.

[36]  C.-C. Jay Kuo,et al.  Spatial and Temporal Error Concealment Techniques for Video Transmission Over Noisy Channels , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Dong-Sik Jang,et al.  Gradual shot boundary detection using localized edge blocks , 2006, Multimedia Tools and Applications.

[38]  Somnath Sengupta,et al.  Hybrid temporal/spatial error concealment strategy robust to scene transitions , 2011, Proceedings of 2011 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing.

[39]  Xiaojing Ma,et al.  A Data Hiding Algorithm for H.264/AVC Video Streams Without Intra-Frame Distortion Drift , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Jae-Young Pyun Error concealment aware streaming video system over packet-based mobile networks , 2008, IEEE Transactions on Consumer Electronics.

[41]  Jechang Jeong,et al.  Fine directional interpolation for spatial error concealment , 2006, IEEE Transactions on Consumer Electronics.

[42]  Yukinobu Taniguchi,et al.  PanoramaExcerpts: extracting and packing panoramas for video browsing , 1997, MULTIMEDIA '97.