Varifocal Occlusion-Capable Optical See-through Augmented Reality Display based on Focus-tunable Optics

Optical see-through augmented reality (AR) systems are a next-generation computing platform that offer unprecedented user experiences by seamlessly combining physical and digital content. Many of the traditional challenges of these displays have been significantly improved over the last few years, but AR experiences offered by today's systems are far from seamless and perceptually realistic. Mutually consistent occlusions between physical and digital objects are typically not supported. When mutual occlusion is supported, it is only supported for a fixed depth. We propose a new optical see-through AR display system that renders mutual occlusion in a depth-dependent, perceptually realistic manner. To this end, we introduce varifocal occlusion displays based on focus-tunable optics, which comprise a varifocal lens system and spatial light modulators that enable depth-corrected hard-edge occlusions for AR experiences. We derive formal optimization methods and closed-form solutions for driving this tunable lens system and demonstrate a monocular varifocal occlusion-capable optical see-through AR display capable of perceptually realistic occlusion across a large depth range.

[1]  Gordon Wetzstein,et al.  Consistent illumination within optical see-through augmented environments , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[2]  Holger Regenbrecht,et al.  ChromaGlasses: Computational Glasses for Compensating Colour Blindness , 2018, CHI.

[3]  Gordon Wetzstein,et al.  Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays , 2017, Proceedings of the National Academy of Sciences.

[4]  Ian P. Howard,et al.  Seeing in Depth , 2008 .

[5]  M. Ernst,et al.  Focus cues affect perceived depth. , 2005, Journal of vision.

[6]  James E. Cutting,et al.  Perceiving Layout and Knowing Distances , 1995 .

[7]  Douglas Lanman,et al.  Pinlight displays , 2014, ACM Trans. Graph..

[8]  Henry Fuchs,et al.  Computational augmented reality eyeglasses , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[9]  Mark Billinghurst,et al.  An occlusion capable optical see-through head mount display for supporting co-located collaboration , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[10]  Christopher D. Saunter,et al.  Dynamic lens and monovision 3D displays to improve viewer comfort , 2015, Optics express.

[11]  Hiroyuki Ohno,et al.  An optical see-through display for mutual occlusion with a real-time stereovision system , 2001, Comput. Graph..

[12]  Hong Hua,et al.  Design and prototype of an augmented reality display with per-pixel mutual occlusion capability. , 2017, Optics express.

[13]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[14]  Gordon Wetzstein,et al.  Tensor displays , 2012, ACM Trans. Graph..

[15]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[16]  Henry Fuchs,et al.  General-purpose telepresence with head-worn optical see-through displays and projector-based lighting , 2013, 2013 IEEE Virtual Reality (VR).

[17]  Hiroyuki Ohno,et al.  An optical see-through display for mutual occlusion of real and virtual environments , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[18]  Karol Myszkowski,et al.  Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors , 2017, IEEE Transactions on Visualization and Computer Graphics.

[19]  H. Barlow.  Vision Science: Photons to Phenomenology by Stephen E. Palmer , 2000, Trends in Cognitive Sciences.

[20]  Quinn Smithwick,et al.  Perspective correct occlusion-capable augmented reality displays using cloaking optics constraints , 2017 .

[21]  Daisuke Iwai,et al.  Light Attenuation Display: Subtractive See-Through Near-Eye Display via Spatial Color Filtering , 2019, IEEE Transactions on Visualization and Computer Graphics.

[22]  Gordon Wetzstein,et al.  Autofocals: Evaluating gaze-contingent eyeglasses for presbyopes , 2019, Science Advances.

[23]  Yuta Itoh,et al.  Occlusion Leak Compensation for Optical See-Through Displays Using a Single-Layer Transmissive Spatial Light Modulator , 2017, IEEE Transactions on Visualization and Computer Graphics.

[24]  Henry Fuchs,et al.  Real-world occlusion in optical see-through AR displays , 2017, VRST.

[25]  Ozan Cakmakci,et al.  Design of a compact optical see-through head-worn display with mutual occlusion capability , 2005, SPIE Optics + Photonics.

[26]  Holger Regenbrecht,et al.  Real-Time Radiometric Compensation for Optical See-Through Head-Mounted Displays , 2016, IEEE Transactions on Visualization and Computer Graphics.

[27]  Gordon Wetzstein,et al.  The visual computing of projector-camera systems , 2008, SIGGRAPH '08.

[28]  Christian Sandor,et al.  BrightView: Increasing Perceived Brightness of Optical See-Through Head-Mounted Displays Through Unnoticeable Incident Light Reduction , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[29]  Pierre-Yves Laffont,et al.  Verifocal: a platform for vision correction and accommodation in head-mounted displays , 2018, SIGGRAPH Emerging Technologies.

[30]  Douglas Lanman,et al.  Content-adaptive parallax barriers: optimizing dual-layer 3D displays using low-rank light field factorization , 2010, ACM Trans. Graph..

[31]  Campbell Fw.  A method for measuring the depth of field of the human eye. , 1954 .

[32]  Joseph S. Choi,et al.  Paraxial ray optics cloaking , 2015, Photonics West - Optoelectronic Materials and Devices.

[33]  Fumio Kishino,et al.  Proposal for a 3‐D display with accommodative compensation: 3DDAC , 1996 .

[34]  Sheng Liu,et al.  An optical see-through head mounted display with addressable focal planes , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[35]  Peter Shirley,et al.  Near-eye varifocal augmented reality display using see-through screens , 2017, ACM Trans. Graph..

[36]  Gordon Wetzstein,et al.  Optical Image Processing Using Light Modulation Displays , 2010, Comput. Graph. Forum.

[37]  Yasuhiro Takaki,et al.  See-through Integral Imaging Display with Background Occlusion Capability , 2015 .

[38]  Alexander Toet,et al.  Visual comfort of binocular and 3D displays , 2004 .

[39]  Yuta Itoh,et al.  Varifocal Occlusion for Optical See-Through Head-Mounted Displays using a Slide Occlusion Mask , 2019, IEEE Transactions on Visualization and Computer Graphics.

[40]  Jannick P. Rolland,et al.  A compact optical see-through head-worn display with occlusion support , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[41]  Hong Hua,et al.  Occlusion capable optical see-through head-mounted display using freeform optics , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[42]  Gordon Wetzstein,et al.  Novel Optical Configurations for Virtual Reality: Evaluating User Preference and Performance with Focus-tunable and Monovision Near-eye Displays , 2016, CHI.

[43]  Henry Fuchs,et al.  FocusAR: Auto-focus Augmented Reality Eyeglasses for both Real World and Virtual Imagery , 2018, IEEE Transactions on Visualization and Computer Graphics.

[44]  Hong Hua,et al.  Optical see-through head-mounted display with occlusion capability , 2013, Defense, Security, and Sensing.

[45]  Oliver Bimber,et al.  Occlusion shadows: using projected light to generate realistic occlusion effects for view-dependent optical see-through displays , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[46]  Henry Fuchs,et al.  An Extended Depth-at-Field Volumetric Near-Eye Augmented Reality Display , 2018, IEEE Transactions on Visualization and Computer Graphics.