Foveated Retinal Optimization for See-Through Near-Eye Multi-Layer Displays

In order to implement 3-D displays with focus cues, several technologies, including multi-layer displays, have been introduced and studied. In multi-layer displays, a volumetric 3-D scene is represented by 2-D layer images via optimization process. Although this methodology has been thoroughly explored and discussed in optical aspect, the optimization method has not been fully analyzed. In this paper, we deal with pupil movement that may prevent efficient synthesis of layer images. We propose a novel optimization method called foveated retinal optimization, which considers the foveated visual acuity of human. Exploiting the characteristic of human vision, our method has tolerance for pupil movement without gaze tracking while maintaining image definition and accurate focus cues. We demonstrate and verify our method in terms of contrast, visual metric, and experimental results. In experiment, we implement a see-through near-eye display that consists of two display modules, a light guide, and a holographic lens. The holographic lens enables us to design a more compact prototype as performing the roles of an image combiner and floating lens, simultaneously. Our system achieves $38^\circ \times 19^\circ $ field of view, continuous focus cues, low aberration, small form factor, and clear see-through property.

[1]  Desney S. Tan,et al.  Foveated 3D graphics , 2012, ACM Trans. Graph..

[2]  Byoungho Lee,et al.  Computational multi-projection display. , 2016, Optics express.

[3]  Byoungho Lee,et al.  Additive light field displays , 2016, ACM Trans. Graph..

[4]  James F. O'Brien,et al.  Optimal presentation of imagery with focus cues on multi-plane displays , 2015, ACM Trans. Graph..

[5]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[6]  Douglas Lanman,et al.  Pinlight displays , 2014, ACM Trans. Graph..

[7]  Henry Fuchs,et al.  Computational augmented reality eyeglasses , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[8]  Sheng Liu,et al.  An optical see-through head mounted display with addressable focal planes , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[9]  Lawrence Stark,et al.  Control of human eye movements: III. dynamic characteristics of the eye tracking mechanism , 1974 .

[10]  J. Goodman Introduction to Fourier optics , 1969 .

[11]  Gordon Wetzstein,et al.  Novel Optical Configurations for Virtual Reality: Evaluating User Preference and Performance with Focus-tunable and Monovision Near-eye Displays , 2016, CHI.

[12]  Sung-Wook Min,et al.  Integral floating display systems for augmented reality. , 2012, Applied optics.

[13]  James Gao,et al.  High-speed switchable lens enables the development of a volumetric stereoscopic display. , 2009, Optics express.

[14]  Byoungho Lee,et al.  Holographic display for see-through augmented reality using mirror-lens holographic optical element. , 2016, Optics letters.

[15]  Byoungho Lee,et al.  Compact three-dimensional head-mounted display system with Savart plate. , 2016, Optics express.

[16]  Changwon Jang,et al.  Retinal 3D , 2017, ACM Trans. Graph..

[17]  Katsuyuki Akutsu,et al.  A full‐color eyewear display using planar waveguides with reflection volume holograms , 2009 .

[18]  Gordon Wetzstein,et al.  Accommodation-invariant computational near-eye displays , 2017, ACM Trans. Graph..

[19]  Kevin J. MacKenzie,et al.  Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control. , 2010, Journal of vision.

[20]  Katsuyuki Akutsu,et al.  A full-color eyewear display using planar waveguides with reflection volume holograms , 2009 .

[21]  Douglas Lanman,et al.  Content-adaptive parallax barriers: optimizing dual-layer 3D displays using low-rank light field factorization , 2010, ACM Trans. Graph..

[22]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, SIGGRAPH 2011.

[23]  Ichiro KASAI,et al.  A Practical See-Through Head Mounted Display Using a Holographic Optical Element , 2001 .

[24]  Byoungho Lee,et al.  Analysis and Implementation of Hologram Lenses for See-Through Head-Mounted Display , 2017, IEEE Photonics Technology Letters.

[25]  Martin S. Banks,et al.  Creating effective focus cues in multi-plane 3D displays , 2011, Optics express.

[26]  Hideya Takahashi,et al.  Stereoscopic see-through retinal projection head-mounted display , 2008, Electronic Imaging.

[27]  A. Kak,et al.  Simultaneous Algebraic Reconstruction Technique (SART): A Superior Implementation of the Art Algorithm , 1984, Ultrasonic imaging.

[28]  Pat Hanrahan,et al.  Achieving Near-Correct Focus Cues Using Multiple Image Planes , 2004 .

[29]  Gordon Wetzstein,et al.  Focus 3D: Compressive accommodation display , 2013, TOGS.

[30]  Douglas Lanman,et al.  Focal surface displays , 2017, ACM Trans. Graph..

[31]  Jian Yang,et al.  Design and fabrication of a holographic head-up display with asymmetric field of view. , 2014, Applied optics.

[32]  Sheng Liu,et al.  A systematic method for designing depth-fused multi-focal plane three-dimensional displays. , 2010, Optics express.

[33]  Joohwan Kim,et al.  Towards foveated rendering for gaze-tracked virtual reality , 2016, ACM Trans. Graph..

[34]  Takanori Senoh,et al.  Projection-type see-through holographic three-dimensional display , 2016, Nature Communications.

[35]  Byoungho Lee,et al.  See-through optical combiner for augmented reality head-mounted display: index-matched anisotropic crystal lens , 2017, Scientific Reports.

[36]  Xu Liu,et al.  Near eye light field display based on human visual features. , 2017, Optics express.

[37]  I. Rentschler,et al.  Peripheral vision and pattern recognition: a review. , 2011, Journal of vision.

[38]  Jae-Hyeung Park,et al.  3D holographic head mounted display using holographic optical elements with astigmatism aberration compensation. , 2015, Optics express.

[39]  Yang Li,et al.  Design of see-through near-eye display for presbyopia. , 2017, Optics express.

[40]  Martin S. Banks,et al.  A stereo display prototype with multiple focal distances , 2004, SIGGRAPH 2004.

[41]  Gordon Wetzstein,et al.  The light field stereoscope , 2015, ACM Trans. Graph..

[42]  Hong Hua,et al.  High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics. , 2014, Optics express.

[43]  Marc Levoy,et al.  Gaze-directed volume rendering , 1990, I3D '90.

[44]  David M. Hoffman,et al.  The zone of comfort: Predicting visual discomfort with stereo displays. , 2011, Journal of vision.