Enhancement of Wireless 3D Video Communication Using Color-Plus-Depth Error Restoration Algorithms and Bayesian Kalman Filtering

This paper proposes a hybrid of Outer Block Boundary Matching Algorithm and Directional Interpolation Error Concealment Algorithm (DIECA) to recover the Motion Vectors (MVs) and the Disparity Vectors (DVs) of the lost color frames of the transmitted Three-Dimensional Video (3DV). For the lost 3DV depth frames, an Encoder-Independent Decoder-Dependent Depth-Assisted Error Concealment (EIDD-DAEC) algorithm is proposed. It exploits the recovered color MVs and DVs to estimate more additional concealment depth-assisted MVs and DVs. After that, the initially-estimated concealment candidate DVs and MVs are selected from all previously-predicted ones using the DIECA and the Decoder Motion Vector Estimation Algorithm (DMVEA). Finally, the proposed Bayesian Kalman Filtering (BKF) scheme is efficiently employed to filter out the inherent errors inside the selected concealment candidate color-plus-depth MVs and DVs to achieve better 3DV quality. Extensive experimental results on different standardized 3DV sequences demonstrate that the proposed color-plus-depth schemes are more robust against heavy losses and they achieve high 3DV quality performance with an improved average Peak Signal-to-Noise Ratio (PSNR) gain. They objectively and subjectively outperform the state-of-the-art error recovery techniques, especially at severe Packet Loss Rates (PLRs).

[1]  Yuan Zhou,et al.  Frame Loss Concealment for Multiview Video Transmission Over Wireless Multimedia Sensor Networks , 2015, IEEE Sensors Journal.

[2]  Walid El-Shafai,et al.  Pixel-level Matching Based Multi-hypothesis Error Concealment Modes for Wireless 3D H.264/MVC Communication , 2015 .

[3]  Chaminda T. E. R. Hewage,et al.  Quality of experience for 3D video streaming , 2013, IEEE Communications Magazine.

[4]  Ying Chen,et al.  Next-Generation 3D Formats with Depth Map Support , 2014, IEEE MultiMedia.

[5]  Wen-Nung Lie,et al.  Motion Vector Recovery for Video Error Concealment by Using Iterative Dynamic-Programming Optimization , 2014, IEEE Transactions on Multimedia.

[6]  Wen-Nung Lie,et al.  Error concealment for 3D video transmission , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[7]  Wei Xiang,et al.  Robust Multiview Three-Dimensional Video Communications Based on Distributed Video Coding , 2017, IEEE Systems Journal.

[8]  Wen Gao,et al.  A Novel Error Concealment Method for Stereoscopic Video Coding , 2007, 2007 IEEE International Conference on Image Processing.

[9]  Yusheng Ji,et al.  Optimizing Distributed Source Coding for Interactive Multiview Video Streaming Over Lossy Networks , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Dinh Trieu Duong,et al.  Hybrid Temporal Error Concealment Methods for Block-Based Compressed Video Transmission , 2008, IEEE Transactions on Broadcasting.

[11]  Pascal Frossard,et al.  Optimizing Multiview Video Plus Depth Prediction Structures for Interactive Multiview Video Streaming , 2015, IEEE Journal of Selected Topics in Signal Processing.

[12]  Yan Zhang,et al.  Fast Multiview Video Coding Using Adaptive Prediction Structure and Hierarchical Mode Decision , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Chuen-Ching Wang,et al.  Object-Based Error Concealment in 3D Video , 2011, 2011 Fifth International Conference on Genetic and Evolutionary Computing.

[14]  Jin Wang,et al.  Depth Image-Based Temporal Error Concealment for 3-D Video Transmission , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Ahmet M. Kondoz,et al.  Adaptive delivery of immersive 3D multi-view video over the Internet , 2016, Multimedia Tools and Applications.

[16]  Bogdan Ionescu,et al.  Multiview Plus Depth Video Coding With Temporal Prediction View Synthesis , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Jan Mochnac,et al.  Kalman filter based error concealment algorithm , 2009 .

[18]  Wen-Nung Lie,et al.  Video error concealment by using Kalman-filtering technique , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[19]  Huijuan Cui,et al.  An effective error concealment scheme for heavily corrupted H.264/AVC videos based on Kalman filtering , 2014, Signal Image Video Process..

[20]  Edward J. Delp,et al.  Spatial subsampling-based multiple description video coding with adaptive temporal-spatial error concealment , 2015, 2015 Picture Coding Symposium (PCS).

[21]  Chang-Su Kim,et al.  Frame loss concealment for stereoscopic video plus depth sequences , 2011, IEEE Transactions on Consumer Electronics.

[22]  Thomas Maugey,et al.  Temporal and Inter-View Consistent Error Concealment Technique for Multiview Plus Depth Video , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Christine Guillemot,et al.  Video Inpainting With Short-Term Windows: Application to Object Removal and Error Concealment , 2015, IEEE Transactions on Image Processing.

[24]  Bo Yan,et al.  Efficient Frame Concealment for Depth Image-Based 3-D Video Transmission , 2012, IEEE Transactions on Multimedia.

[25]  Kuei-Ting Kuo,et al.  An Adaptive Error Concealment Method Based on Fuzzy Reasoning for Multi-View Video Coding , 2014, Journal of Display Technology.