Vision enhancement: defocus correction via optical see-through head-mounted displays

Vision is our primary, essential sense to perceive the real world. Human beings have been keen to enhance the limit of the eye function by inventing various vision devices such as corrective glasses, sunglasses, telescopes, and night vision goggles. Recently, Optical See-Through Head-Mounted Displays (OST-HMD) have penetrated in the commercial market. While the traditional devices have improved our vision by altering or replacing it, OST-HMDs can augment and mediate it. We believe that future OST-HMDs will dramatically improve our vision capability, combined with wearable sensing systems including image sensors. For taking a step toward this future, this paper investigates Vision Enhancement (VE) techniques via OST-HMDs. We aim at correcting optical defects of human eyes, especially defocus, by overlaying a compensation image on the user's actual view so that the filter cancels the aberration. Our contributions are threefold. Firstly, we formulate our method by taking the optical relationships between OST-HMD and human eye into consideration. Secondly, we demonstrate the method in proof-of-concept experiments. Lastly and most importantly, we provide a thorough analysis of the results including limitations of the current system, potential research issues necessary for realizing practical VE systems, and possible solutions for the issues for future research.

[1]  Meenakshisundaram Gopi,et al.  Edge-guided resolution enhancement in projectors via optical pixel sharing , 2012, ACM Trans. Graph..

[2]  R. Shack,et al.  History and principles of Shack-Hartmann wavefront sensing. , 2001, Journal of refractive surgery.

[3]  Hironobu Fujiyoshi,et al.  Acquisition of 3D gaze information from eyeball movements using inside-out camera , 2011, AH '11.

[4]  E Peli,et al.  Vision Multiplexing: an Engineering Approach to Vision Rehabilitation Device Development , 2001, Optometry and vision science : official publication of the American Academy of Optometry.

[5]  Luca Maria Gambardella,et al.  Artificial Defocus for Displaying Markers in Microscopy Z-Stacks , 2011, IEEE Transactions on Visualization and Computer Graphics.

[6]  Atsushi Nakazawa,et al.  Corneal Imaging Revisited: An Overview of Corneal Reflection Analysis and Applications , 2013, IPSJ Trans. Comput. Vis. Appl..

[7]  Gordon Wetzstein,et al.  The Visual Computing of Projector‐Camera Systems , 2008, SIGGRAPH '08.

[8]  G Westheimer,et al.  The Maxwellian view. , 1966, Vision research.

[9]  Mike Bennett,et al.  Creating personalized digital human models of perception for visual analytics , 2011, UMAP'11.

[10]  Gordon Wetzstein,et al.  Computational light field display for correcting visual aberrations , 2013, SIGGRAPH '13.

[11]  Junzhong Liang,et al.  Objective measurement of wave aberrations of the human eye with the use of a Hartmann-Shack wave-front sensor. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  M. Shikida,et al.  Novel high resolution optical scanner actuated by aerosol deposited PZT films , 2003, The Sixteenth Annual International Conference on Micro Electro Mechanical Systems, 2003. MEMS-03 Kyoto. IEEE.

[13]  David R. Flatla,et al.  Color correction for optical see-through displays using display color profiles , 2013, VRST '13.

[14]  Daniel G. Aliaga,et al.  Tailored displays to compensate for visual aberrations , 2012, ACM Trans. Graph..

[15]  Ross T. Smith,et al.  Adaptive substrate for enhanced spatial augmented reality contrast and resolution , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[16]  Marco Lombardo,et al.  Wave aberration of human eyes and new descriptors of image optical quality and visual performance. , 2010, Journal of cataract and refractive surgery.

[17]  Supun Samarasekera,et al.  Augmented Reality Binoculars , 2013, IEEE Transactions on Visualization and Computer Graphics.

[18]  S. Ando,et al.  Super-resolution reconstruction algorithm using vibro-imaging and correlation image sensor , 2012, 2012 Proceedings of SICE Annual Conference (SICE).

[19]  Eli Peli,et al.  Treating with Spectacle Lenses: A Novel Idea!? , 2002, Optometry and vision science : official publication of the American Academy of Optometry.

[20]  Oliver Bimber,et al.  Superimposing dynamic range , 2008, SIGGRAPH Asia '08.

[21]  Hirokazu Kato,et al.  Appearance control by projector camera feedback for visually impaired , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[22]  Pourang Irani,et al.  SmartColor: Real-time color correction and contrast for optical see-through head-mounted displays , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[23]  Rajaraman Suryakumar,et al.  Application of video-based technology for the simultaneous measurement of accommodation and vergence , 2007, Vision Research.

[24]  Vitor F. Pamplona,et al.  NETRA: interactive display for estimating refractive errors and focal range , 2010 .

[25]  Tobias Höllerer,et al.  A hand-held AR magic lens with user-perspective rendering , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[26]  Alex D. Hwang,et al.  An Augmented-Reality Edge Enhancement Application for Google Glass , 2014, Optometry and vision science : official publication of the American Academy of Optometry.

[27]  Kosuke Sato,et al.  Approximated user-perspective rendering in tablet-based augmented reality , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[28]  Masahiko Inami,et al.  Stop motion goggle: augmented visual perception by subtraction method using high speed liquid crystal , 2012, AH '12.

[29]  Gudrun Klinker,et al.  Interaction-free calibration for optical see-through head-mounted displays based on 3D Eye localization , 2014, 2014 IEEE Symposium on 3D User Interfaces (3DUI).

[30]  Christian Sandor,et al.  Improving Spatial Perception for Augmented Reality X-Ray Vision , 2009, 2009 IEEE Virtual Reality Conference.

[31]  Hideo Saito,et al.  Defocus Blur Correcting Projector-Camera System , 2008, ACIVS.