Novel Pixel Recovery Method Based on Motion Vector Disparity and Compensation Difference

As compressed videos are transmitted in the communication networks, video packet loss inevitably occurs. This problem can be solved by error concealment method. We used the motion vector of the available neighboring blocks to estimate the lost motion vector for the lost block. These estimates propagate to predict all other missing motion vectors. We further improved the work by using the idea of the motion vector disparities between neighboring available blocks to modify the motion vector weightings. Furthermore, the differences between the compensated pixels and the decoded pixels in the neighboring blocks are computed for another weighting for improvement. These two novelties are combined as a final indicator to prediction weightings. By comparison against the state-of-the-art method, the four proposed algorithms increase the average peak signal-to-noise ratio (PSNR) by up to 1.86, 1.93, 1.94, and 2.04 dB on average, showing the gradual improvement of our design systems. For other video quality measurements, the average gains of the proposed work against the state-of-the-art work can be up to 0.0575 in structural similarity index metric (SSIM), −0.0278 in video quality metric (VQM) (the lower the better), −0.0008 in motion-based video integrity evaluation (MOVIE) (the lower the better), and 2.77 in subjective evaluation. The proposed work performs slightly worse than a pixel-based state-of-the-art method in PSNR and SSIM but performs better in VQM and MOVIE (both correlate better with human perception) and subjective experiments, with much lower computational complexity.

[1]  Ting-Lan Lin,et al.  Video Motion Vector Recovery Method Using Decoding Partition Information , 2016, Journal of Display Technology.

[2]  Shih-Chang Hsia An edge-oriented spatial interpolation for consecutive block error concealment , 2004, IEEE Signal Process. Lett..

[3]  Bo Yan,et al.  Efficient Motion Vector Interpolation for Error Concealment of H.264/AVC , 2011, IEEE Transactions on Broadcasting.

[4]  Enrico Magli,et al.  Concealment of whole-frame losses for wireless low bit-rate video based on multiframe optical flow estimation , 2005, IEEE Transactions on Multimedia.

[5]  V. Kamakoti,et al.  Efficient Motion Vector Recovery Algorithm for H.264 Using B-Spline Approximation , 2010, IEEE Transactions on Broadcasting.

[6]  Cedric Nishan Canagarajah,et al.  Enhanced error concealment with mode selection , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Liu Da The Error Concealment Feature in the H.26L Test Model , 2003 .

[8]  Yanling Xu,et al.  H.264 video communication based refined error concealment schemes , 2004, IEEE Transactions on Consumer Electronics.

[9]  Guangtao Zhai,et al.  Spatial Error Concealment With an Adaptive Linear Predictor , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Ting-Lan Lin,et al.  Video error concealment method using motion vector estimation propagation , 2017, 2017 International Conference on Applied System Innovation (ICASI).

[11]  Oscar C. Au,et al.  Video Error Concealment Using Spatio-Temporal Boundary Matching and Partial Differential Equation , 2008, IEEE Transactions on Multimedia.

[12]  Bertrand Granado,et al.  Sparse Recovery-Based Error Concealment , 2017, IEEE Transactions on Multimedia.

[13]  Gwo-Long Li,et al.  Effective error concealment algorithm of whole frame loss for H.264 video coding standard by recursive motion vector refinement , 2010, IEEE Transactions on Consumer Electronics.

[14]  Qiang Peng,et al.  Block-based temporal error concealment for video packet using motion vector extrapolation , 2002, IEEE 2002 International Conference on Communications, Circuits and Systems and West Sino Expositions.

[15]  Dinh Trieu Duong,et al.  Hybrid Temporal Error Concealment Methods for Block-Based Compressed Video Transmission , 2008, IEEE Transactions on Broadcasting.

[16]  Weisi Lin,et al.  Bayesian Error Concealment With DCT Pyramid for Images , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[18]  Yo-Sung Ho,et al.  Error concealment technique based on optical flow , 2002 .

[19]  Huijun Gao,et al.  Sparsity-Based Image Error Concealment via Adaptive Dual Dictionary Learning and Regularization , 2017, IEEE Transactions on Image Processing.

[20]  Wen Gao,et al.  Packet Video Error Concealment With Auto Regressive Model , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Bo Yan,et al.  A Hybrid Frame Concealment Algorithm for H.264/AVC , 2010, IEEE Transactions on Image Processing.

[22]  Wen-Chih Chen,et al.  Recovery of Lost Motion Vectors Using Encoded Residual Signals , 2013, IEEE Transactions on Broadcasting.

[23]  Jun Wu,et al.  A temporal error concealment method for H.264/AVC using motion vector recovery , 2008, IEEE Transactions on Consumer Electronics.

[24]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..