Perceptually Driven Nonuniform Asymmetric Coding of Stereoscopic 3D Video

Asymmetric stereoscopic video coding has already proven its effectiveness in reducing the bandwidth required for stereoscopic 3D delivery without degrading the visual quality. This approach, in which the left and right views are encoded with different levels of quality, relies on the perceptual theory of binocular suppression. However, to ensure comfortable 3D viewing, the just-noticeable level of asymmetry, i.e., the maximum quality gap between views, has to be carefully defined. Both subjectively and empirically fixed thresholds of asymmetry demonstrated either the maladjustment to content or dependency to the experimental design. This paper describes a new nonuniform asymmetric stereoscopic video coding method adaptively adjusting the level of asymmetry for each region of the image based on its perceptual significance. The proposed method uses a fully automated model that dynamically determines the best bounds of asymmetry for which the 3D viewing experience will not be altered. This is achieved by exploiting several human-visual-system-inspired models, namely, the binocular just-noticeable difference, and the visual saliency map and depth information. The simulation results show that the proposed method results in bit rate saving of up to 26% and provides better 3D visual quality compared with state-of-the-art asymmetric coding methods.

[1]  Namho Hur,et al.  Asymmetric Coding of Stereoscopic Video for Transmission Over T-DMB , 2007, 2007 3DTV Conference.

[2]  Ken Chen,et al.  Stereoscopic video coding with asymmetric luminance and chrominance qualities , 2010, IEEE Transactions on Consumer Electronics.

[3]  Aljoscha Smolic,et al.  Efficient Prediction Structures for Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Michael G. Perkins,et al.  Data compression of stereopairs , 1992, IEEE Trans. Commun..

[5]  A. Murat Tekalp,et al.  Temporal and spatial scaling for stereoscopic video compression , 2006, 2006 14th European Signal Processing Conference.

[6]  J. Holmes,et al.  The effect of induced monocular blur on measures of stereoacuity. , 2009, Journal of AAPOS : the official publication of the American Association for Pediatric Ophthalmology and Strabismus.

[7]  Judith Redi,et al.  Quantifying the importance of preserving video quality in visually important regions at the expense of background content , 2015, Signal Process. Image Commun..

[8]  I C Wood,et al.  STEREOPSIS WITH SPATIALLY‐DEGRADED IMAGES * , 1983, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[9]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[10]  Thomas Wiegand,et al.  Mixed resolution coding of stereoscopic video for Mobile devices , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[11]  A. Murat Tekalp,et al.  Evaluation of Asymmetric Stereo Video Coding and Rate Scaling for Adaptive 3D Video Streaming , 2011, IEEE Transactions on Broadcasting.

[12]  André Vincent,et al.  Stereo image quality: effects of mixed spatio-temporal resolution , 2000, IEEE Trans. Circuits Syst. Video Technol..

[13]  Chang Hong Liu,et al.  Differential binocular input and local stereopsis , 2003, Vision Research.

[14]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[16]  S. Coren,et al.  Lateral preferences and human behavior , 1981 .

[17]  Miska M. Hannuksela,et al.  Subjective quality assessment of asymmetric stereoscopic 3D video , 2015, Signal Image Video Process..

[18]  S. Fezza,et al.  Color Correction for Stereo and Multi-view Coding , 2015 .

[19]  Truong Q. Nguyen,et al.  Comparing Perceived Quality and Fatigue for Two Methods of Mixed Resolution Stereoscopic Coding , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Lina J. Karam,et al.  Increased depth perception with sharpness enhancement for stereo video , 2010, Electronic Imaging.

[21]  Miska M. Hannuksela,et al.  Impact of downsampling ratio in mixed-resolution stereoscopic video , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[22]  Ying Chen,et al.  Low-complexity asymmetric multiview video coding , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[23]  S. McKee,et al.  Stereoscopic acuity with defocused and spatially filtered retinal images , 1980 .

[24]  Zhenzhong Chen,et al.  Binocular Just-Noticeable-Difference Model for Stereoscopic Images , 2011, IEEE Signal Processing Letters.

[25]  Miska M. Hannuksela,et al.  Asymmetrie luminance based filtering for stereoscopic video compression , 2014, 2014 International Conference on 3D Imaging (IC3D).

[26]  B. Julesz Foundations of Cyclopean Perception , 1971 .

[27]  Tao Chen,et al.  3D-TV Content Storage and Transmission , 2011, IEEE Transactions on Broadcasting.

[28]  Qingxiong Yang,et al.  Recursive Bilateral Filtering , 2012, ECCV.

[29]  Touradj Ebrahimi,et al.  Calculation of average coding efficiency based on subjective quality scores , 2014, Journal of Visual Communication and Image Representation.

[30]  Panos Nasiopoulos,et al.  Effect of eye dominance on the perception of stereoscopic 3D video , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[31]  Heiko Schwarz,et al.  3D High-Efficiency Video Coding for Multi-View Video and Depth Data , 2013, IEEE Transactions on Image Processing.

[32]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, SIGGRAPH 2011.

[33]  Irena Koprinska,et al.  Temporal video segmentation: A survey , 2001, Signal Process. Image Commun..

[34]  Ahmet M. Kondoz,et al.  Visual Attention Model Aided Non-Uniform Asymmetric Coding of Stereoscopic Video , 2014, IEEE Journal of Selected Topics in Signal Processing.

[35]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[36]  Wijnand A. IJsselsteijn,et al.  Perceived quality of compressed stereoscopic images: Effects of symmetric and asymmetric JPEG coding and camera separation , 2006, TAP.

[37]  Munchurl Kim,et al.  Binocular Suppression-Based Stereoscopic Video Coding by Joint Rate Control With KKT Conditions for a Hybrid Video Codec System , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[38]  Azeddine Beghdadi,et al.  A survey of perceptual image processing methods , 2013, Signal Process. Image Commun..

[39]  Kamel Mohamed Faraoun,et al.  Feature-Based Color Correction of Multiview Video for Coding and Rendering Enhancement , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Ankit K. Jain,et al.  Apparent sharpness of 3D video when one eye's view is more blurry , 2013, i-Perception.

[41]  Ahmet M. Kondoz,et al.  Psycho-physical limits of interocular blur suppression and its application to asymmetric stereoscopic video delivery , 2012, 2012 19th International Packet Video Workshop (PV).

[42]  D V Meegan,et al.  Unequal weighting of monocular inputs in binocular combination: implications for the compression of stereoscopic imagery. , 2001, Journal of experimental psychology. Applied.