Gaze-contingent ocular parallax rendering for virtual reality

Immersive computer graphics systems strive to generate perceptually realistic user experiences. Current-generation virtual reality (VR) displays are successful in accurately rendering many perceptually important effects, including perspective, disparity, motion parallax, and other depth cues. In this article, we introduce ocular parallax rendering, a technology that accurately renders small amounts of gaze-contingent parallax capable of improving depth perception and realism in VR. Ocular parallax describes the small amounts of depth-dependent image shifts on the retina that are created as the eye rotates. The effect occurs because the centers of rotation and projection of the eye are not the same. We study the perceptual implications of ocular parallax rendering by designing and conducting a series of user experiments. Specifically, we estimate perceptual detection and discrimination thresholds for this effect and demonstrate that it is clearly visible in most VR applications. Additionally, we show that ocular parallax rendering provides an effective ordinal depth cue and it improves the impression of realistic depth in VR.

[1]  Bliss M. Altenhoff,et al.  Effects of calibration to visual and haptic feedback on near-field depth perception in an immersive virtual environment , 2012, SAP.

[2]  Desney S. Tan,et al.  Foveated 3D graphics , 2012, ACM Trans. Graph..

[3]  Suzanne P. McKee,et al.  The spatial requirements for fine stereoacuity , 1983, Vision Research.

[4]  Gordon Wetzstein,et al.  Novel Optical Configurations for Virtual Reality: Evaluating User Preference and Performance with Focus-tunable and Monovision Near-eye Displays , 2016, CHI.

[5]  G. Westheimer,et al.  Effects of practice and the separation of test targets on foveal and peripheral stereoacuity , 1983, Vision Research.

[6]  Karol Myszkowski,et al.  Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors , 2017, IEEE Transactions on Visualization and Computer Graphics.

[7]  Andrew T. Duchowski,et al.  Gaze-Contingent Displays: A Review , 2004, Cyberpsychology Behav. Soc. Netw..

[8]  John D. Pettigrew,et al.  Convergence of specialised behaviour, eye movements and visual optics in the sandlance (Teleostei) and the chameleon (Reptilia) , 1999, Current Biology.

[9]  James F. O'Brien,et al.  Optimal presentation of imagery with focus cues on multi-plane displays , 2015, ACM Trans. Graph..

[10]  Effects of pupil center shift on ocular aberrations. , 2014, Investigative ophthalmology & visual science.

[11]  Christopher D. Saunter,et al.  Dynamic lens and monovision 3D displays to improve viewer comfort , 2015, Optics express.

[12]  Bahram Javidi,et al.  A 3D integral imaging optical see-through head-mounted display. , 2014, Optics express.

[13]  LIII. On the law of visible position in single and binocular vision, and on the representation of solid figures by the union of dissimilar plane pictures on the retina , 1844 .

[14]  Boris M. Velichkovsky,et al.  The perception of egocentric distances in virtual environments - A review , 2013, ACM Comput. Surv..

[15]  Geoffrey P. Bingham,et al.  Optical flow from eye movement with head immobilized: “Ocular occlusion” beyond the nose , 1993, Vision Research.

[16]  Nikhil Balram,et al.  Design and optimization of a near-eye multifocal display system for augmented reality , 2015 .

[17]  Yifan Peng,et al.  Holographic near-eye displays based on overlap-add stereograms , 2019, ACM Trans. Graph..

[18]  Bliss M. Altenhoff,et al.  Near-field distance perception in real and virtual environments using both verbal and action responses , 2011, TAP.

[19]  Andreas Georgiou,et al.  Holographic near-eye displays for virtual and augmented reality , 2017, ACM Trans. Graph..

[20]  W. B. Thompson,et al.  Relative motion: Kinetic information for the order of depth at an edge , 1987, Perception & psychophysics.

[21]  University of California,et al.  Correct Blur and Accommodation Information Is a Reliable Cue to Depth Ordering , .

[22]  Hiroshi Ono,et al.  The rhino-optical phenomenon: Ocular parallax and the visible field beyond the nose , 1986, Vision Research.

[23]  Felix A. Wichmann,et al.  Painfree and accurate Bayesian estimation of psychometric functions for (potentially) overdispersed data , 2016, Vision Research.

[24]  Donald H. House,et al.  Reducing visual discomfort of 3D stereoscopic displays with gaze-contingent depth-of-field , 2014, SAP.

[25]  Gordon Wetzstein,et al.  The light field stereoscope , 2015, ACM Trans. Graph..

[26]  M. Ernst,et al.  Focus cues affect perceived depth. , 2005, Journal of vision.

[27]  Alexei A. Goon,et al.  Multifocal planes head-mounted displays. , 2000, Applied optics.

[28]  Miguel A. Nacenta,et al.  Depth perception with gaze-contingent depth of field , 2014, CHI.

[29]  G. Westheimer.  Eye movement responses to a horizontally moving visual stimulus. , 1954, A.M.A. archives of ophthalmology.

[30]  Rahul Narain,et al.  Blur and the perception of depth at occlusions. , 2016, Journal of vision.

[31]  David M. Hoffman,et al.  Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. , 2008, Journal of vision.

[32]  Douglas Lanman,et al.  Fast gaze-contingent optimal decompositions for multifocal displays , 2017, ACM Trans. Graph..

[33]  Jessika Weiss,et al.  Vision Science Photons To Phenomenology , 2016 .

[34]  Noboru Ohnishi,et al.  Effect of the sight line shift when a head-mounted display is used , 2000, Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143).

[35]  S. McKee,et al.  The detection of motion in the peripheral visual field , 1984, Vision Research.

[36]  Peter Shirley,et al.  Fundamentals of computer graphics , 2018 .

[37]  Gordon D. Love,et al.  Chromablur , 2017, ACM Trans. Graph..

[38]  Mark Nawrot,et al.  Eye movements provide the extra-retinal signal required for the perception of depth from motion parallax , 2003, Vision Research.

[39]  Sheng Liu,et al.  An optical see-through head mounted display with addressable focal planes , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[40]  Michael F. Land,et al.  Fast-focus telephoto eye , 1995, Nature.

[41]  Erik Blaser,et al.  Retinal blur and the perception of egocentric distance. , 2010, Journal of vision.

[42]  G A FRY,et al.  The center of rotation of the eye. , 1962, American journal of optometry and archives of American Academy of Optometry.

[43]  Hong Hua,et al.  High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics. , 2014, Optics express.

[44]  James Gao,et al.  High-speed switchable lens enables the development of a volumetric stereoscopic display. , 2009, Optics express.

[45]  Noboru Ohnishi,et al.  Study on the ocular parallax as a monocular depth cue induced by small eye movements during a gaze , 1998, Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vol.20 Biomedical Engineering Towards the Year 2000 and Beyond (Cat. No.98CH36286).

[46]  O. Braddick,et al.  Seeing in Depth , 2008 .

[47]  S. Palmer,et al.  Edge-region grouping in figure-ground organization and depth perception. , 2008, Journal of experimental psychology. Human perception and performance.

[48]  Douglas Lanman,et al.  Near-eye light field displays , 2013, SIGGRAPH '13.

[49]  F A Wichmann,et al.  Ning for Helpful Comments and Suggestions. This Paper Benefited Con- Siderably from Conscientious Peer Review, and We Thank Our Reviewers the Psychometric Function: I. Fitting, Sampling, and Goodness of Fit , 2001 .

[50]  Felix Wichmann,et al.  The psychometric function: I , 2001 .

[51]  Martin S. Banks,et al.  A stereo display prototype with multiple focal distances , 2004, ACM Trans. Graph..

[52]  G Ishai,et al.  Visual stability and space perception in monocular vision: mathematical model. , 1980, Journal of the Optical Society of America.

[53]  T Shipley,et al.  Stereoscopic acuity and horizontal angular distance from fixation. , 1969, Journal of the Optical Society of America.

[54]  Joohwan Kim,et al.  Towards foveated rendering for gaze-tracked virtual reality , 2016, ACM Trans. Graph..

[55]  Gordon Wetzstein,et al.  Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays , 2017, Proceedings of the National Academy of Sciences.

[56]  G. Mather,et al.  Blur Discrimination and its Relation to Blur-Mediated Depth Perception , 2002, Perception.

[57]  K N OGLE,et al.  Depth of focus of the human eye. , 1959, Journal of the Optical Society of America.

[58]  Andrew T. Smith,et al.  Visual detection of motion , 1994 .

[59]  Jack M. Loomis,et al.  Visual perception of egocentric distance in real and virtual environments. , 2003 .

[60]  Felix Wichmann,et al.  The psychometric function: II. Bootstrap-based confidence intervals and sampling , 2001, Perception & psychophysics.

[61]  C. A. Burbeck,et al.  Occlusion edge blur: a cue to relative visual depth. , 1996, Journal of the Optical Society of America. A, Optics, image science, and vision.

[62]  H. Kudo,et al.  Measurement of the ability in monocular depth perception during gazing at near visual target-effect of the ocular parallax cue , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[63]  Hans-Peter Seidel,et al.  A perceptual model for disparity , 2011, ACM Trans. Graph..