Multidimensional characterization of quality of experience of stereoscopic 3D TV. (Caractérisation multidimensionnelle de la qualité d'expérience en télévision de la TV3D stéréoscopique)

Stereoscopic-3DTV (S-3DTV) should provide enhanced depth perception to viewer while it might affect other factors of user experience. Subjective assessment is the most direct way to assess quality of experience (QoE). However, conventional assessment methods are not sufficient to evaluate the QoE of S-3DTV. This thesis aims first to propose new methodologies to evaluate S-3DTV QoE; second, investigate different technical issues related to QoE along the 3DTV broadcasting chain; third, propose recommendations to optimize the S-3DTV QoE. For methodological aspects, the key idea relies on using multidimensional QoE indicators. Visual fatigue, as a particular dimension of QoE, is addressed separately under optimized viewing conditions using questionnaire, vision test and EEG signals. For other QoE indicators, we design subjective QoE experiments to investigate the impact of content acquisition, 3D representation format, compression and transmission on QoE of S-3DTV. The experiment results quantitatively reveal how perceived binocular depth, compression distortion, the cooperation between 3D representation formats and line interleaved display, and view asymmetries affect multidimensional QoE of S-3DTV. Additionally, we elaborate and validate improved stereoscopic shooting rules, depth budget for visual comfort, appropriate frame compatible format for line interleaved display, bitrate to broadcast S-3DTV, threshold for view asymmetries to avoid visual discomfort.

[1]  Chris Varekamp,et al.  Enabling Introduction of Stereoscopic (3D) Video: Formats and Compression Standards , 2007, 2007 IEEE International Conference on Image Processing.

[2]  Aljoscha Smolic,et al.  An overview of available and emerging 3D video formats and depth enhanced stereo as efficient generic solution , 2009, 2009 Picture Coding Symposium.

[3]  Fabio Gagliardi Cozman,et al.  Depth from scattering , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  A. Gotchev,et al.  Classification of stereoscopic artefacts , 2009 .

[5]  J R Jiménez,et al.  Associated Phoria in Relation to Stereopsis with Random-Dot Stereograms , 2000, Optometry and vision science : official publication of the American Academy of Optometry.

[6]  R. Hays,et al.  Development of the 25-item National Eye Institute Visual Function Questionnaire. , 2001, Archives of ophthalmology.

[7]  L. Morin,et al.  Representation and coding of 3D video data , 2010 .

[8]  Wijnand A. IJsselsteijn,et al.  Viewing experience and naturalness of 3D images , 2005, SPIE Optics East.

[9]  S Nagata,et al.  The binocular fusion of human vision on stereoscopic displays--field of view and environment effects. , 1996, Ergonomics.

[10]  Charles A. Poynton,et al.  Rehabilitation of gamma , 1998, Electronic Imaging.

[11]  Robert Patterson,et al.  Human factors of 3‐D displays , 2007 .

[12]  Masaki Emoto,et al.  Changes in fusional vergence limit and its hysteresis after viewing stereoscopic TV , 2004 .

[13]  ウェブ、スチュアート,et al.  High-definition multimedia interface , 2007 .

[14]  Wijnand A. IJsselsteijn,et al.  Subjective evaluation of stereoscopic images: effects of camera parameters and display duration , 2000, IEEE Trans. Circuits Syst. Video Technol..

[15]  Wijnand A. IJsselsteijn,et al.  Perceptual attributes of crosstalk in 3D images , 2005, Displays.

[16]  Takashi Shibata,et al.  Visual discomfort with stereo displays: effects of viewing distance and direction of vergence-accommodation conflict , 2011, Electronic Imaging.

[17]  Makoto Okui,et al.  Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV images , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Sumio Yano,et al.  A study of visual fatigue and visual comfort for 3D HDTV/HDTV images , 2002 .

[19]  Neil A. Dodgson,et al.  Autostereoscopic 3D displays , 2005, Computer.

[20]  S. Pastoor.  Human factors of 3DTV: an overview of current research at Heinrich-Hertz-Institut Berlin , 1992 .

[21]  Samuel R. Buss,et al.  3D Computer Graphics: A Mathematical Introduction with OpenGL , 2003 .

[22]  Peter A. Howarth,et al.  Visual fatigue caused by viewing stereoscopic motion images: Background, theories, and observations , 2008, Displays.

[23]  D M Levi,et al.  Binocular summation in vernier acuity. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[24]  James E. Cutting,et al.  Chapter 3 – Perceiving Layout and Knowing Distances: The Integration, Relative Potency, and Contextual Use of Different Information about Depth* , 1995 .

[25]  Masayuki Tanimoto,et al.  Multiview Imaging and 3DTV , 2007, IEEE Signal Processing Magazine.

[26]  Neil A. Dodgson,et al.  Variation and extrema of human interpupillary distance , 2004, IS&T/SPIE Electronic Imaging.

[27]  Jens Ogniewski,et al.  Autostereoscopy and Motion Parallax for Mobile Computer Games Using Commercially Available Hardware , 2011, CISIM 2011.

[28]  Christopher W. Tyler,et al.  Spatio-temporal properties of Panum's fusional area , 1981, Vision Research.

[29]  Herman B. K. Boom,et al.  Bridging Disciplines For Biomedicine , 1997 .

[30]  Wijnand A. IJsselsteijn,et al.  Perceived quality of compressed stereoscopic images: Effects of symmetric and asymmetric JPEG coding and camera separation , 2006, TAP.

[31]  Ma Conway,et al.  Handbook of perception and cognition , 1996 .

[32]  Korbinian Brodmann,et al.  Brodmann's localization in the cerebral cortex : the principles of comparative localisation in the cerebral cortex based on cytoarchitectonics , 2006 .

[33]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Kenneth J. Ciuffreda,et al.  Vergence eye movements : basic and clinical aspects , 1983 .

[35]  Bernard Mendiburu,et al.  3D Movie Making: Stereoscopic Digital Cinema from Script to Screen , 2009 .

[36]  Masaki Emoto,et al.  Two factors in visual fatigue caused by stereoscopic HDTV images , 2004 .

[37]  I. Yuyama,et al.  Recent Advances in HDTV - Present State and Future Development of HDTV- , 1991, The Second International Conference on Image Management and Communication (IMAC) in Patient Care: New Technologies for Better Patient Care,.

[38]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[39]  M. Ardito,et al.  Influence of display parameters on perceived HDTV quality , 1996 .

[40]  Ray Zone.  Stereoscopic Cinema and the Origins of 3-D Film, 1838-1952 , 2007 .

[41]  Y. Yeh,et al.  Limits of Fusion and Depth Judgment in Stereoscopic Color Displays , 1990, Human factors.

[42]  A. Murat Tekalp,et al.  Evaluation of Asymmetric Stereo Video Coding and Rate Scaling for Adaptive 3D Video Streaming , 2011, IEEE Transactions on Broadcasting.

[43]  Wijnand A. IJsselsteijn,et al.  A survey of perceptual quality issues in three-dimensional television systems , 2003, IS&T/SPIE Electronic Imaging.

[44]  Yong Ju Jung,et al.  A novel 2D-to-3D conversion technique based on relative height-depth cue , 2009, Electronic Imaging.

[45]  R. Granit.  THE HEART ( Extract from “ Principles and Applications of Bioelectric and Biomagnetic Fields , 2005 .

[46]  Wijnand A. IJsselsteijn,et al.  Evaluation of Stereoscopic Images: Beyond 2D Quality , 2011, IEEE Transactions on Broadcasting.

[47]  G. Iddan,et al.  3D IMAGING IN THE STUDIO (AND ELSEWHERE...) , 2001 .

[48]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[49]  C. Fehn.  A 3D-TV system based on video plus depth information , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[50]  C. Schor,et al.  Disparity range for local stereopsis as a function of luminance spatial frequency , 1983, Vision Research.

[51]  Rajaraman Suryakumar,et al.  Study of the dynamic interactions between vergence and accommodation , 2005 .

[52]  O. Braddick,et al.  Seeing in Depth , 2008 .

[53]  Andrew J. Woods,et al.  Image distortions in stereoscopic video systems , 1993, Electronic Imaging.

[54]  Methodology for the subjective assessment of video quality in multimedia applications ( Question , 2007 .

[55]  Matthias Wöpking,et al.  Viewing comfort with stereoscopic pictures : an experimental study on the subjective effects of disparity magnitude and depth of focus , 1995 .

[56]  Thierry Alpert,et al.  Human factor requirements for a stereoscopic television service: admissible contrast differences between the two channels of a stereoscopic camera , 1994, Electronic Imaging.

[57]  Patrick Le Callet,et al.  Influence of depth rendering on the quality of experience for an autostereoscopic display , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[58]  Keetaek Kham,et al.  Measurement of 3D Visual Fatigue Using Event-Related Potential (ERP): 3D Oddball Paradigm , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[59]  Hirokazu Yamanoue.  The Differences Between Toed-in Camera Configurations and Parallel Camera Configurations in Shooting Stereoscopic Images , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[60]  T. Wiegand,et al.  Efficient Compression of Multi-View Depth Data Based on MVC , 2007, 2007 3DTV Conference.

[61]  Ashutosh Saxena,et al.  Learning Depth from Single Monocular Images , 2005, NIPS.

[62]  E. A. Hendriks,et al.  Converting 2D to 3D: A Survey , 2005 .

[63]  Benny Bing.  3D and HD Broadband Video Networking , 2010 .

[64]  Pierre Boher,et al.  Full optical characterization of autostereoscopic 3D displays using local viewing angle and imaging measurements , 2012, Electronic Imaging.

[65]  Yoshio Nakashima,et al.  Wavelength difference limit for binocular color fusion , 1980, Vision Research.

[66]  Wijnand A. IJsselsteijn,et al.  A survey of perceptual evaluations and requirements of three-dimensional TV , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[67]  Jun Okamoto,et al.  Subjective characteristics for stereoscopic high definition video , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[68]  Aljoscha Smolic,et al.  Scene Representation Technologies for 3DTV—A Survey , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[69]  Charles Wheatstone.  On some remarkable and hitherto unobserved phenomena of binocular vision. , 1962 .

[70]  Thomas Wiegand,et al.  Mixed resolution coding of stereoscopic video for Mobile devices , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[71]  Djemel Ziou,et al.  Depth from Defocus Estimation in Spatial Domain , 2001, Comput. Vis. Image Underst..

[72]  Masayuki Tanimoto.  Overview of free viewpoint television , 2006, Signal Process. Image Commun..

[73]  David M. Hoffman,et al.  The zone of comfort: Predicting visual discomfort with stereo displays. , 2011, Journal of vision.

[74]  Lawrence Stark,et al.  Blur: A sufficient accommodative stimulus , 1977, Documenta Ophthalmologica.

[75]  Aljoscha Smolic,et al.  Efficient Prediction Structures for Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[76]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[77]  C. Wheatstone.  XVIII. Contributions to the physiology of vision. —Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision , 1962, Philosophical Transactions of the Royal Society of London.

[78]  František Mikšícek.  Causes of Visual Fatigue and Its Improvements in Stereoscopy , 2006 .

[79]  Yi-Ping Hung,et al.  Video-based eye tracking for autostereoscopic displays , 2001 .

[80]  Marc Lambooij.  Stereoscopic displays and visual comfort: a review , 2007 .

[81]  Hirokazu Yamanoue,et al.  Orthostereoscopic conditions for 3D HDTV , 1998, Electronic Imaging.

[82]  Theo Lasser,et al.  Multiplexed 3D imaging using wavelength encoded spectral interferometry: a proof of principle , 2003 .

[83]  Jun Chen,et al.  A Simple Semi-automatic Technique for 2D to 3D Video Conversion , 2011, AICI.

[84]  B. Girod,et al.  Multiview Video Compression , 2007, IEEE Signal Processing Magazine.

[85]  V. V. Krishnan,et al.  Disparity vergence: a proposed name for a dominant component of binocular vergence eye movements. , 1980, American journal of optometry and physiological optics.

[86]  I. Heynderickx,et al.  Measuring visual discomfort associated with 3D displays , 2009, Electronic Imaging.

[87]  Alexander Toet,et al.  Visual comfort of binocular and 3D displays , 2004 .

[88]  Touradj Ebrahimi,et al.  A comprehensive database and subjective evaluation methodology for quality of experience in stereoscopic video , 2010, Electronic Imaging.

[89]  Charles Wheatstone,et al.  Contributions to the Physiology of Vision. , 1837 .

[90]  Thrasyvoulos N. Pappas,et al.  Perceptual criteria for image quality evaluation , 2005 .

[91]  Jens-Rainer Ohm,et al.  Stereo/multiview video encoding using the MPEG family of standards , 1999, Electronic Imaging.

[92]  Pjh Pieter Seuntiëns,et al.  Visual experience of 3D TV , 2006 .

[93]  Touradj Ebrahimi,et al.  Temporal synchronization in stereoscopic video: Influence on quality of experience and automatic asynchrony detection , 2010, 2010 IEEE International Conference on Image Processing.

[94]  Graham R Jones,et al.  Controlling perceived depth in stereoscopic images , 2001, IS&T/SPIE Electronic Imaging.

[95]  T. Takeda,et al.  Evaluation of stereoscopic display task environment by measurement of accommodation responses , 1996, Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[96]  J. Kuze,et al.  Subjective evaluation of visual fatigue caused by motion images , 2008, Displays.

[97]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[98]  Christoph Fehn,et al.  3D TV Broadcasting , 2006 .

[99]  Andrew J. Woods,et al.  Understanding Crosstalk in Stereoscopic Displays , 2010 .

[100]  H Ohzu,et al.  Accommodative responses to stereoscopic three-dimensional display. , 1997, Applied optics.

[101]  Larry F. Hodges,et al.  Geometric Considerations for Stereoscopic Virtual Environments , 1993, Presence: Teleoperators & Virtual Environments.

[102]  Jerry D. Gibson,et al.  Handbook of Image and Video Processing , 2000 .

[103]  Paul Milgram,et al.  Perceptual issues in augmented reality , 1996, Electronic Imaging.

[104]  Angeline M. Loh,et al.  Shape from Non-homogeneous, Non-stationary, Anisotropic, Perspective Texture , 2005, BMVC.

[105]  Nicolas S. Holliman,et al.  Mapping perceived depth to regions of interest in stereoscopic images , 2004, IS&T/SPIE Electronic Imaging.

[106]  Hyun Wook Park,et al.  A robust stereo disparity estimation using adaptive window search and dynamic programming search , 2001, Pattern Recognit..

[107]  G Westheimer,et al.  Population distribution of stereoscopic ability , 1993, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[108]  Whitman Richards,et al.  A Planar and a Volumetric Test for Stereoanomaly , 2002, Perception.

[109]  Iain E. G. Richardson,et al.  H.264 and MPEG-4 Video Compression: Video Coding for Next-Generation Multimedia , 2003 .

[110]  D.B. Diner.  A new definition of orthostereopsis for 3D television , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[111]  Gary J. Sullivan,et al.  Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard , 2011, Proceedings of the IEEE.

[112]  Karen O. Egiazarian,et al.  Classification and simulation of stereoscopic artifacts in mobile 3DTV content , 2009, Electronic Imaging.

[113]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[114]  Cesson Sévigné.  NEW QUALITY EVALUATION METHOD SUITED TO MULTIMEDIA CONTEXT SAMVIQ , 2006 .

[115]  Christine Guillemot,et al.  Incremental-LDI for multi-view coding , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[116]  Touradj Ebrahimi,et al.  Impact of Acquisition Distortion on the Quality of Stereoscopic Images , 2010 .

[117]  H. Kalva,et al.  Evaluation of 3DTV Service using Asymmetric View Coding Based on MPEG-2 , 2007, 2007 3DTV Conference.

[118]  Arthur D. Broadbent,et al.  A critical review of the development of the CIE1931 RGB color-matching functions , 2004 .

[119]  Qingming Huang,et al.  Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model , 2009, Signal Process. Image Commun..

[120]  Sebastiano Battiato,et al.  Depth map generation by image classification , 2004, IS&T/SPIE Electronic Imaging.

[121]  Andrew J. Woods.  Optimal usage of LCD projectors for polarized stereoscopic projection , 2001, IS&T/SPIE Electronic Imaging.

[122]  Aljoscha Smolic,et al.  Coding Algorithms for 3DTV—A Survey , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[123]  Hyung-Chul O. Li,et al.  Method of Measuring Subjective 3D Visual Fatigue: A Five-Factor Model , 2008 .

[124]  Whitman Richards,et al.  Stereopsis and stereoblindness , 2006, Experimental Brain Research.