Efficient Multiview Video Coding Using 3-D Coding and Saliency-Based Bit Allocation

Capturing a scene using multiple cameras from different angles is expected to provide the necessary interactivity in the 3-D space to satisfy end-users’ demands for observing objects and actions from different angles and depths. Existing multiview video coding (MVC) technologies face tradeoff among rate-distortion performance, random access frame delay, i.e., interactivity, and computational time. To address above mentioned tradeoffs, a novel cuboid MVC strategy is proposed with 3-D frame referencing structure to improve interactivity and computational time, an additional reference frame to improve rate-distortion performance for occluded areas, and visual attention-based bit allocation to provide better perceptual video quality. The experimental results reveal that the proposed scheme provides better interactivity, reduced computational time, and better perceptual quality compared to the 3D-HEVC implementation, HTM 15.0.

[1]  Wen Gao,et al.  Fast disparity and motion estimation based on correlations for multiview video coding , 2008, IEEE Transactions on Consumer Electronics.

[2]  Dar-Shyang Lee,et al.  Effective Gaussian mixture learning for video background subtraction , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Pamela C. Cosman,et al.  Selection of Long-Term Reference Frames in Dual-Frame Video Coding Using Simulated Annealing , 2008, IEEE Signal Processing Letters.

[4]  Gary J. Sullivan,et al.  Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard , 2011, Proceedings of the IEEE.

[5]  Christine Guillemot,et al.  Perceptually-Friendly H.264/AVC Video Coding Based on Foveated Just-Noticeable-Distortion Model , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Bu-Sung Lee,et al.  A Long-Term Reference Frame for Hierarchical B-Picture-Based Video Coding , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Cedric Nishan Canagarajah,et al.  Towards efficient context-specific video coding based on gaze-tracking analysis , 2007, TOMCCAP.

[8]  Jerry D. Gibson,et al.  Distributions of 3D DCT coefficients for video , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[9]  Wen Gao,et al.  Dual Frame Motion Compensation With Optimal Long-Term Reference Frame Selection and Bit Allocation , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Wolfgang Fichtner,et al.  A 3D-DCT real-time video compression system for low complexity single-chip VLSI implementation , 2000 .

[11]  Pamela C. Cosman,et al.  Dual Frame Motion Compensation With Uneven Quality Assignment , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Janusz Konrad,et al.  Motion analysis in 3D DCT domain and its application to video coding , 2005, Signal Process. Image Commun..

[13]  Laurent Itti,et al.  Visual attention guided bit allocation in video compression , 2011, Image Vis. Comput..

[14]  Manoranjan Paul,et al.  Fast Intermode Selection for HEVC Video Coding Using Phase Correlation , 2014, 2014 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[15]  André Kaup,et al.  Analysis of Multi-Reference Block Matching for MultiView Video Coding , 2006 .

[16]  Li Li,et al.  Multiview video compression with 3D-DCT , 2007, 2007 ITI 5th International Conference on Information and Communications Technology.

[17]  Manoranjan Paul,et al.  A hybrid object detection technique from dynamic background using Gaussian mixture models , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[18]  Bu-Sung Lee,et al.  McFIS in hierarchical bipredictve pictures-based video coding for referencing the stable area in a scene , 2011, 2011 18th IEEE International Conference on Image Processing.

[19]  Christof Koch,et al.  Image Signature: Highlighting Sparse Salient Regions , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Markus Flierl,et al.  Generalized B pictures and the draft H.264/AVC video-compression standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[21]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[22]  Touradj Ebrahimi,et al.  Towards high efficiency video coding: Subjective evaluation of potential coding technologies , 2011, J. Vis. Commun. Image Represent..

[23]  Manoranjan Paul,et al.  On Stable Dynamic Background Generation Technique Using Gaussian Mixture Models for Robust Object Detection , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[24]  Mohamed-Chaker Larabi,et al.  Attentional mechanisms driven adaptive quantization and selective bit allocation scheme for H.264/AVC , 2013, Signal Process. Image Commun..

[25]  Dietmar Hepper,et al.  Efficiency analysis and application of uncovered background prediction in a low bit rate image coder , 1990, IEEE Trans. Commun..

[26]  Manoranjan Paul,et al.  Video Coding Focusing on Block Partitioning and Occlusion , 2010, IEEE Transactions on Image Processing.

[27]  Boon-Lock Yeo,et al.  Volume Rendering of DCT-Based Compressed 3D Scalar Data , 1995, IEEE Trans. Vis. Comput. Graph..

[28]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[29]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[30]  Hai Liu,et al.  A Novel Scheme of Multi-View Video Coding for Low-Delay View Random Access , 2010, 2010 5th International Conference on Future Information Technology.

[31]  J. Li,et al.  An Epipolar Geometry-Based Fast Disparity Estimation Algorithm for Multiview Image and Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Qian Huang,et al.  An efficient coding scheme for surveillance videos captured by stationary cameras , 2010, Visual Communications and Image Processing.

[33]  Manoranjan Paul,et al.  Disparity-adjusted 3D multi-view video coding with dynamic background modelling , 2013, 2013 IEEE International Conference on Image Processing.

[34]  Bu-Sung Lee,et al.  Explore and Model Better I-Frames for Video Coding , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Pascal Frossard,et al.  Optimizing Multiview Video Plus Depth Prediction Structures for Interactive Multiview Video Streaming , 2015, IEEE Journal of Selected Topics in Signal Processing.

[36]  Mahsa Talebpourazad,et al.  3D-TV Content generation and multi-view video coding , 2010 .

[37]  Yun Zhang,et al.  Rate Distortion Optimized Inter-View Frame Level Bit Allocation Method for MV-HEVC , 2015, IEEE Transactions on Multimedia.

[38]  Wen Gao,et al.  An efficient foreground-based surveillance video coding scheme in low bit-rate compression , 2012, 2012 Visual Communications and Image Processing.

[39]  Bernd Girod,et al.  Background extraction and long-term memory motion-compensated prediction for spatial-random-access-enabled video coding , 2009, 2009 Picture Coding Symposium.

[40]  Qionghai Dai,et al.  Background-frame based motion compensation for video compression , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[41]  Josef Kittler,et al.  A background memory update scheme for H.263 video codec , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[42]  Liang-Gee Chen,et al.  Efficient moving object segmentation algorithm using background registration technique , 2002, IEEE Trans. Circuits Syst. Video Technol..

[43]  Naoki Mukawa,et al.  Uncovered Background Prediction in Interframe Coding , 1985, IEEE Trans. Commun..

[44]  Zhou Wang,et al.  Embedded foveation image coding , 2001, IEEE Trans. Image Process..

[45]  Michael R. Frater,et al.  An Efficient Mode Selection Prior to the Actual Encoding for H.264/AVC Encoder , 2009, IEEE Transactions on Multimedia.

[46]  Xianguo Zhang,et al.  Low-complexity and high-efficiency background modeling for surveillance video coding , 2012, 2012 Visual Communications and Image Processing.

[47]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[48]  Wen Gao,et al.  Low-delay View Random Access for Multi-view Video Coding , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[49]  Josef Kittler,et al.  Using background memory for efficient video coding , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[50]  Heiko Schwarz,et al.  3D High-Efficiency Video Coding for Multi-View Video and Depth Data , 2013, IEEE Transactions on Image Processing.

[51]  Bu-Sung Lee,et al.  Direct Intermode Selection for H.264 Video Coding Using Phase Correlation , 2011, IEEE Transactions on Image Processing.

[52]  Bu-Sung Lee,et al.  Video coding with dynamic background , 2013, EURASIP J. Adv. Signal Process..

[53]  Simone Milani,et al.  A novel multi-view image coding scheme based on view-warping and 3D-DCT , 2010, J. Vis. Commun. Image Represent..