Modeling and optimizing eye vergence response to stereoscopic cuts

Sudden temporal depth changes, such as cuts that are introduced by video edits, can significantly degrade the quality of stereoscopic content. Since usually not encountered in the real world, they are very challenging for the audience. This is because the eye vergence has to constantly adapt to new disparities in spite of conflicting accommodation requirements. Such rapid disparity changes may lead to confusion, reduced understanding of the scene, and overall attractiveness of the content. In most cases the problem cannot be solved by simply matching the depth around the transition, as this would require flattening the scene completely. To better understand this limitation of the human visual system, we conducted a series of eye-tracking experiments. The data obtained allowed us to derive and evaluate a model describing adaptation of vergence to disparity changes on a stereoscopic display. Besides computing user-specific models, we also estimated parameters of an average observer model. This enables a range of strategies for minimizing the adaptation time in the audience.

[1]  G WESTHEIMER,et al.  Factors influencing accommodation responses of the human eye. , 1959, Journal of the Optical Society of America.

[2]  L Stark,et al.  AN ANALYSIS OF LATENCIES AND PREDICTION IN THE FUSIONAL VERGENCE SYSTEM* , 1973, American journal of optometry and archives of American Academy of Optometry.

[3]  DOUGLAS SHIRACHI,et al.  Dynamic Measures of Vergence Accommodation , 1977, American journal of optometry and physiological optics.

[4]  J Semmlow,et al.  Dynamic contributions of the components of binocular vergence. , 1979, Journal of the Optical Society of America.

[5]  Clifton M. Schor,et al.  The relationship between fusional vergence eye movements and fixation disparity , 1979, Vision Research.

[6]  A. Watson,et al.  Quest: A Bayesian adaptive psychometric method , 1983, Perception & psychophysics.

[7]  G K Hung,et al.  Quantitative assessment of disparity vergence components. , 1986, Investigative ophthalmology & visual science.

[8]  Edward A. Lee,et al.  Static Scheduling of Synchronous Data Flow Programs for Digital Signal Processing , 1989, IEEE Transactions on Computers.

[9]  H Collewijn,et al.  Ocular vergence under natural conditions. II. Gaze shifts between real targets differing in distance and direction , 1989, Proceedings of the Royal Society of London. B. Biological Sciences.

[10]  R. Finke,et al.  Principles of mental imagery , 1989 .

[11]  G K Hung Adaptation model of accommodation and vergence , 1992, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[12]  C. Schor,et al.  A dynamic model of cross-coupling between accommodation and convergence: simulations of step and frequency responses. , 1992, Optometry and vision science : official publication of the American Academy of Optometry.

[13]  G K Hung,et al.  Vergence eye movements under natural viewing conditions. , 1994, Investigative ophthalmology & visual science.

[14]  F. Okuyama Human visual accommodation and vergence eye movement while viewing stereoscopic display and actual target , 1998, Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vol.20 Biomedical Engineering Towards the Year 2000 and Beyond (Cat. No.98CH36286).

[15]  G K Hung,et al.  Dynamic model of the vergence eye movement system: simulations using MATLAB/SIMULINK. , 1998, Computer methods and programs in biomedicine.

[16]  Clifton Schor,et al.  The influence of interactions between accommodation and convergence on the lag of accommodation , 1999, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[17]  S K Rushton,et al.  Developing visual systems and exposure to virtual reality and stereo displays: some concerns and speculations about the demands on accommodation and vergence. , 1999, Applied ergonomics.

[18]  A S Eadie,et al.  Modelling adaptation effects in vergence and accommodation after exposure to a simulated Virtual Reality stimulus , 2000, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[19]  George K. Hung Models of Oculomotor Control , 2001 .

[20]  Yumi Kato,et al.  The use of video refraction to measure the dynamic properties of the near triad in observers of a 3‐D display , 2002, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[21]  Wijnand A. IJsselsteijn,et al.  A survey of perceptual evaluations and requirements of three-dimensional TV , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[23]  Masaki Emoto,et al.  Two factors in visual fatigue caused by stereoscopic HDTV images , 2004 .

[24]  Tara L. Alvarez,et al.  Divergence eye movements are dependent on initial stimulus position , 2005, Vision Research.

[25]  Frédo Durand,et al.  Antialiasing for automultiscopic 3D displays , 2006, EGSR '06.

[26]  L. Itti,et al.  Visual causes versus correlates of attentional selection in dynamic scenes , 2006, Vision Research.

[27]  David M. Hoffman,et al.  Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. , 2008, Journal of vision.

[28]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[29]  Bernard Mendiburu,et al.  3D Movie Making: Stereoscopic Digital Cinema from Script to Screen , 2009 .

[30]  John M. Henderson,et al.  Clustering of Gaze During Dynamic Scene Viewing is Predicted by Motion , 2011, Cognitive Computation.

[31]  M. Gross,et al.  Nonlinear disparity mapping for stereoscopic 3D , 2010, ACM Trans. Graph..

[32]  Markus H. Gross,et al.  OSCAM - optimized stereoscopic camera control for interactive 3D , 2011, ACM Trans. Graph..

[33]  Michael F. Cohen,et al.  A Viewer-Centric Editor for 3D Movies , 2011, IEEE Computer Graphics and Applications.

[34]  Wijnand A. IJsselsteijn,et al.  Visual discomfort of 3D TV: Assessment methods and modeling , 2011, Displays.

[35]  Peter Kauff,et al.  Production Rules for Stereo Acquisition , 2011, Proceedings of the IEEE.

[36]  Kaitlin L. Brunick,et al.  Quicker, faster, darker: Changes in Hollywood film over 75 years , 2011, i-Perception.

[37]  David M. Hoffman,et al.  The zone of comfort: Predicting visual discomfort with stereo displays. , 2011, Journal of vision.

[38]  Aljoscha Smolic,et al.  Computational stereo camera system with programmable control loop , 2011, ACM Trans. Graph..

[39]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Mark Horowitz,et al.  Avoiding game over: Bringing design to the next level , 2012, DAC Design Automation Conference 2012.

[41]  Filippo Speranza,et al.  Visual comfort: stereoscopic objects moving in the horizontal and mid-sagittal planes , 2012, Electronic Imaging.

[42]  Uri Hasson,et al.  Temporal eye movement strategies during naturalistic viewing. , 2012, Journal of vision.

[43]  Jan Vitek,et al.  Terra: a multi-stage language for high-performance computing , 2013, PLDI.

[44]  Diego Gutierrez,et al.  A metric of visual comfort for stereoscopic motion , 2013, ACM Trans. Graph..

[45]  Efstathios Stavrakis,et al.  The effects of fast disparity adjustment in gaze-controlled stereoscopic applications , 2014, ETRA.