Retinal image quality in near-eye pupil-steered systems.

State-of-the-art near-eye displays often compromise on eye box size to maintain a wide field of view, necessitating a means for steering the eye box to maintain alignment with a moving eye. The design space of such pupil-steered systems is not well defined and the implications of imperfect steering on the perceived image are not well understood. To better characterize the pupil steering design space, we introduce a generalized taxonomy of pupil-steered architectures that considers both system and ocular factors that affect steering performance. We also develop an optical model of a generalized pupil-steered system with a wide-field schematic eye to simulate the retinal image. Using this framework, we systematically characterize retinal image quality for different combinations of design parameters. The results of these simulations provide an overview of the pupil steering design space and help determine relevant psychophysical experiments for further evaluation.

[1]  Walter B. Lancaster,et al.  Fifty Years' Experience in Ocular Motility*: Part I , 1941 .

[2]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, SIGGRAPH 2011.

[3]  G. Wyszecki,et al.  Axial chromatic aberration of the human eye. , 1957, Journal of the Optical Society of America.

[4]  A. Bradley,et al.  Statistical variation of aberration structure and image quality in a normal population of healthy eyes. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[5]  Beatriz M. Matesanz,et al.  Influence of background size, luminance and eccentricity on different adaptation mechanisms , 2016, Vision Research.

[6]  A. Hendrickson,et al.  Human photoreceptor topography , 1990, The Journal of comparative neurology.

[7]  G. Westheimer Directional sensitivity of the retina: 75 years of Stiles–Crawford effect , 2008, Proceedings of the Royal Society B: Biological Sciences.

[8]  Kevin J. MacKenzie,et al.  Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control. , 2010, Journal of vision.

[9]  R. Baddeley,et al.  The long and the short of it: Spatial statistics at fixation vary with saccade amplitude and task , 2006, Vision Research.

[10]  Joseph A. Izatt,et al.  Wide-field optical model of the human eye with asymmetrically tilted and decentered lens that reproduces measured ocular aberrations , 2015 .

[11]  A. Georgiou,et al.  Wedge guides and pupil steering for mixed reality , 2018 .

[12]  M. Morrone,et al.  Extraretinal Control of Saccadic Suppression , 2000, The Journal of Neuroscience.

[13]  K. Rayner The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search , 2009, Quarterly journal of experimental psychology.

[14]  Andreas Georgiou,et al.  Holographic near-eye displays for virtual and augmented reality , 2017, ACM Trans. Graph..

[15]  G. W. Beeler,et al.  Visual threshold changes resulting from spontaneous saccadic eye movements. , 1967, Vision research.

[16]  L. Stark,et al.  The main sequence, a tool for studying human eye movements , 1975 .

[17]  Walter B. Lancaster,et al.  Fifty Years' Experience in Ocular Motility* , 1941 .

[18]  Jungjin Lee,et al.  Video Extrapolation Using Neighboring Frames , 2019, ACM Trans. Graph..

[19]  L. Stark,et al.  Most naturally occurring human saccades have magnitudes of 15 degrees or less. , 1975, Investigative ophthalmology.

[20]  Dinesh K. Pai,et al.  Fast ray-tracing of human eye optics on Graphics Processing Units , 2014, Comput. Methods Programs Biomed..

[21]  Hans-Peter Seidel,et al.  Design and volume optimization of space structures , 2017, ACM Trans. Graph..

[22]  D. Marquardt An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[23]  Brenton Keller,et al.  Pupil tracking optical coherence tomography for precise control of pupil entry position. , 2015, Biomedical optics express.

[24]  Jyrki Rovamo,et al.  Neural modulation transfer function of the human visual system at various eccentricities , 1995, Vision Research.

[25]  Kenneth Levenberg A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[26]  LAWRENCE STARK,et al.  Pupil Unrest: An Example of Noise in a Biological Servomechanism , 1958, Nature.

[27]  J. Yellott,et al.  A unified formula for light-adapted pupil size. , 2012, Journal of vision.

[28]  Larry N Thibos,et al.  Customized models of ocular aberrations across the visual field during accommodation. , 2019, Journal of vision.

[29]  G. Holstein,et al.  The influence of rod light and dark adaptation upon rod‐cone interaction. , 1983, The Journal of physiology.

[30]  Harold Metcalf Stiles-Crawford Apodization , 1965 .

[31]  F. Campbell,et al.  Saccadic omission: Why we do not see a grey-out during a saccadic eye movement , 1978, Vision Research.

[32]  W. N. Charman,et al.  The effect of pupil centration and diameter on ocular performance , 1988, Vision Research.

[33]  Jae-Hyeung Park,et al.  Optical see-through holographic near-eye-display with eyebox steering and depth of field control. , 2018, Optics express.

[34]  Changwon Jang,et al.  Retinal 3D , 2017, ACM Trans. Graph..

[35]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[36]  Jiaolong Yang,et al.  Image smoothing via unsupervised learning , 2018, ACM Trans. Graph..

[37]  Yasuhiro Takaki,et al.  Flexible retinal image formation by holographic Maxwellian-view display. , 2018, Optics express.

[38]  Rahul Narain,et al.  Blur and the perception of depth at occlusions. , 2016, Journal of vision.

[39]  V C Smith,et al.  Temporal modulation sensitivity and pulse-detection thresholds for chromatic and luminance perturbations. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[40]  W. Stiles,et al.  The Luminous Efficiency of Rays Entering the Eye Pupil at Different Points , 1933 .

[41]  Lawrence Stark,et al.  A model for nonlinear stochastic behavior of the pupil , 1982, Biological Cybernetics.

[42]  E. Hess,et al.  Pupil Size in Relation to Mental Activity during Simple Problem-Solving , 1964, Science.

[43]  Yasuhiro Takaki,et al.  Super multi-view near-eye display to solve vergence-accommodation conflict. , 2018, Optics express.

[44]  R. Rosén,et al.  Effect of induced transverse chromatic aberration on peripheral vision. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[45]  P. Ward,et al.  THE EFFECT OF SPATIAL FREQUENCY ON STEADY‐STATE ACCOMMODATION , 1987, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[46]  Cheng Yao,et al.  Retinal projection head-mounted display , 2017 .

[47]  G Westheimer,et al.  Editorial: Visual acuity and hyperacuity. , 1975, Investigative ophthalmology.

[48]  Holger Lubatschowski,et al.  Virtual Eye: Retinal Image Visualization of the Human Eye , 1997, IEEE Computer Graphics and Applications.

[49]  Alexei A. Goon,et al.  Multifocal planes head-mounted displays. , 2000, Applied optics.

[50]  S. B. Stevenson,et al.  Dependence of visual suppression on the amplitudes of saccades and blinks , 1986, Vision Research.

[51]  H. Urey,et al.  Diffractive exit-pupil expander for display applications. , 2001, Applied optics.

[52]  S J Anderson,et al.  Peripheral spatial vision: limits imposed by optics, photoreceptors, and receptor pooling. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[53]  Leonard Matin,et al.  Metacontrast and Saccadic Suppression , 1972, Science.

[54]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[55]  Jinwoong Kim,et al.  Real-time pupil tracking backlight system for holographic 3D display (Invited Paper) , 2014 .

[56]  A. Bradley,et al.  The chromatic eye: a new reduced-eye model of ocular chromatic aberration in humans. , 1992, Applied optics.

[57]  E. Rossi,et al.  The relationship between visual resolution and cone spacing in the human fovea , 2009, Nature Neuroscience.

[58]  David J. Mack,et al.  The effect of sampling rate and lowpass filters on saccades – A modeling approach , 2017, Behavior Research Methods.

[59]  Bruce Bridgeman,et al.  A theory of visual stability across saccadic eye movements , 1994, Behavioral and Brain Sciences.

[60]  G Westheimer,et al.  The Maxwellian view. , 1966, Vision research.

[61]  B. Howland,et al.  A subjective method for the measurement of monochromatic aberrations of the eye. , 1977, Journal of the Optical Society of America.

[62]  Hei Cheon Yang,et al.  Horizontal two-phase jet behavior with an annular nozzle ejector in the water tank , 2015, J. Vis..

[63]  J. D. Mollon,et al.  The psychophysics of detecting binocular discrepancies of luminance , 2009, Vision Research.

[64]  Emily A. Cooper,et al.  Stereopsis is adaptive for the natural environment , 2015, Science Advances.

[65]  Joseph A. Izatt,et al.  Pupil Tracking for Real-Time Motion Corrected Anterior Segment Optical Coherence Tomography , 2016, PloS one.

[66]  Byoungho Lee,et al.  Holographic near-eye display with expanded eye-box , 2018, ACM Trans. Graph..

[67]  B G Breitmeyer,et al.  Implications of sustained and transient channels for theories of visual pattern masking, saccadic suppression, and information processing. , 1976, Psychological review.

[68]  M. Alpern,et al.  Vergence and accommodation. V. Pupil size changes associated with changes in accommodative vergence. , 1961, American journal of ophthalmology.

[69]  Guillaume S. Masson,et al.  Motion perception during saccadic eye movements , 2000, Nature Neuroscience.

[70]  L Stark,et al.  Topology of the near response triad , 1990, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[71]  D. Williams,et al.  Monochromatic aberrations of the human eye in a large population. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[72]  T. L. Watson,et al.  All is not lost: Post-saccadic contributions to the perceptual omission of intra-saccadic streaks , 2018, Consciousness and Cognition.

[73]  Eric Castet,et al.  Motion perception of saccade-induced retinal translation , 2002, Proceedings of the National Academy of Sciences of the United States of America.