Evaluation of user-centric optical see-through head-mounted display calibration using a leap motion controller

Advances in optical see-through head-mounted display technology have yielded a number of consumer accessible options, such as the Google Glass and Epson Moverio BT-200, and have paved the way for promising next generation hardware, including the Microsoft HoloLens and Epson Pro BT-2000. The release of consumer devices, though, has also been accompanied by an ever increasing need for standardized optical see-through display calibration procedures easily implemented and performed by researchers, developers, and novice users alike. Automatic calibration techniques offer the possibility for ubiquitous environment independent solutions, un-reliant upon user interaction. These processes, however, require the use of additional eye tracking hardware and algorithms not natively present in current display offerings. User dependent approaches, therefore, remain the only viable option for effective calibration of current generation optical see-through hardware. Inclusion of depth sensors and hand tracking cameras, promised in forthcoming consumer models, offer further potential to improve these manual methods and provide practical intuitive calibration options accessible to a wide user base. In this work, we evaluate the accuracy and precision of manual optical see-through head-mounted display calibration performed using a Leap Motion controller. Both hand and stylus based methods for monocular and stereo procedures are examined, along with several on-screen reticle designs for improving alignment context during calibration. Our study shows, that while enhancing the context of reticles for hand based alignments does yield improved results, Leap Motion calibrations performed with a stylus offer the most accurate and consistent performance, comparable to that found in previous studies for environment-centric routines. In addition, we found that stereo calibration further improved precision in every case. We believe that our findings not only validate the potential of hand and gesture based trackers in facilitating optical see-through calibration methodologies, but also provide a suitable benchmark to help guide future efforts in standardizing calibration practices for user friendly consumer systems.

[1]  Arindam Dey,et al.  An empiric evaluation of confirmation methods for optical see-through head-mounted display calibration , 2011, ISMAR.

[2]  Ji Zhou,et al.  Evaluation of calibration procedures for optical see-through head-mounted displays , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[3]  John Quarles,et al.  Shark punch: A virtual reality game for aquatic rehabilitation , 2015, 2015 IEEE Virtual Reality (VR).

[4]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[5]  Jeremy Sutton.  Air painting with Corel Painter Freestyle and the leap motion controller: a revolutionary new way to paint! , 2013, SIGGRAPH '13.

[6]  Nassir Navab,et al.  Single point active alignment method (SPAAM) for optical see-through HMD calibration for AR , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[7]  Ronald Azuma,et al.  Predictive tracking for augmented reality , 1995 .

[8]  Frank Weichert,et al.  Analysis of the Accuracy and Robustness of the Leap Motion Controller , 2013, Sensors.

[9]  Gudrun Klinker,et al.  Interaction-free calibration for optical see-through head-mounted displays based on 3D Eye localization , 2014, 2014 IEEE Symposium on 3D User Interfaces (3DUI).

[10]  Martin A. Skoglund,et al.  Accuracy of Eyepoint Estimation in Optical See-Through Head-Mounted Displays Using the Single Point Active Alignment Method , 2011, VR 2011.

[11]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[12]  Joze Guna,et al.  Evaluation of Leap Motion Controller with a High Precision Optical Tracking System , 2014, HCI.

[13]  Igor Zubrycki,et al.  Using Integrated Vision Systems: Three Gears and Leap Motion, to Control a 3-finger Dexterous Gripper , 2014, Recent Advances in Automation, Robotics and Measuring Techniques.

[14]  Nassir Navab,et al.  An on-line evaluation system for optical see-through augmented reality , 2004, IEEE Virtual Reality 2004.

[15]  Joze Guna,et al.  An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking , 2014, Sensors.

[16]  Martin A. Skoglund,et al.  Parameter estimation variance of the single point active alignment method in optical see-through head mounted display calibration , 2011, 2011 IEEE Virtual Reality Conference.

[17]  Gudrun Klinker,et al.  Performance and sensitivity analysis of INDICA: INteraction-Free DIsplay CAlibration for Optical See-Through Head-Mounted Displays , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[18]  Holger Regenbrecht,et al.  A leap-supported, hybrid AR interface approach , 2013, OZCHI.

[19]  Christian Sandor,et al.  Subjective Evaluation of a Semi-Automatic Optical See-Through Head-Mounted Display Calibration Technique , 2015, IEEE Transactions on Visualization and Computer Graphics.

[20]  T A Travaglini,et al.  Initial Experiments with the Leap Motion as a User Interface in Robotic Endonasal Surgery. , 2016, Robotics and mechatronics : proceedings of the 4th IFToMM International Symposium on Robotics and Mechatronics. IFToMM International Symposium on Robotics and Mechatronics.

[21]  Nassir Navab,et al.  Optical see-through HMD calibration: a stereo method validated with a video see-through system , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[22]  Jake Araullo,et al.  The Leap Motion controller: a view on sign language , 2013, OZCHI.

[23]  Anders Ynnerman,et al.  Optical See-Through Head Mounted Display Direct Linear Transformation Calibration Robustness in the Presence of User Alignment Noise , 2010 .

[25]  Gudrun Klinker,et al.  Corneal-Imaging Calibration for Optical See-Through Head-Mounted Displays , 2015, IEEE Transactions on Visualization and Computer Graphics.

[26]  Fan Xiao,et al.  Display-relative calibration for optical see-through head-mounted displays , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[27]  Atsushi Nakazawa,et al.  Corneal Imaging Revisited: An Overview of Corneal Reflection Analysis and Applications , 2013, IPSJ Trans. Comput. Vis. Appl..

[28]  Neil A. Dodgson,et al.  Robust real-time pupil tracking in highly off-axis images , 2012, ETRA.

[29]  Nassir Navab,et al.  Practical solutions for calibration of optical see-through devices , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.