Infrared bundle adjusting and clustering method for head-mounted display and Leap Motion calibration

Leap Motion has become widely used due to its ability to recognize intuitive hand gestures or accurate finger positions. Attaching a Leap Motion to a virtual reality head-mounted display (VR HMD) is highly interoperable with virtual objects in virtual reality. However, it is difficult for a virtual reality application to identify the accurate position where the Leap Motion is attached to the HMD. This causes errors in the positions of the actual user’s hands and the virtual hands, which makes the interaction in virtual reality difficult. In this paper, a method that calibrates an output area in VR HMD and a sensing area in Leap Motion is proposed. The difference in the origin coordinate between VR HMD and Leap Motion is derived using the proposed method. The position of the Leap Motion attached to the HMD was determined through an experiment using the proposed calibration technique, and the error was approximately 0.757 cm. Accordingly, it enables more intuitive interactions in virtual reality applications.

[1]  Kiyoshi Kiyokawa,et al.  Corneal imaging in localization and HMD interaction , 2014, ISMAR.

[2]  Kyoungsu Oh,et al.  Interactive Experience Room Using Infrared Sensors and User's Poses , 2017, J. Inf. Process. Syst..

[3]  Henry Fuchs,et al.  Comparison of optical and video see-through, head-mounted displays , 1995, Other Conferences.

[4]  Gerd Bruder,et al.  To touch or not to touch?: comparing 2D touch and 3D mid-air interaction on stereoscopic tabletop surfaces , 2013, SUI '13.

[5]  Jean Ponce,et al.  Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Seongah Chin,et al.  Virtual figure model crafting with VR HMD and Leap Motion , 2017 .

[7]  Ken Hinckley,et al.  A survey of design issues in spatial input , 1994, UIST '94.

[8]  Luc Van Gool,et al.  Real-time 3D hand gesture interaction with a robot for understanding directions from humans , 2011, 2011 RO-MAN.

[9]  Gunhee Kim,et al.  A calibration method for optical see-through head-mounted displays with a depth camera , 2016, 2016 IEEE Virtual Reality (VR).

[10]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[11]  Andrea Colaco.  Sensor design and interaction techniques for gestural input to smart glasses and mobile devices , 2013, UIST '13 Adjunct.

[12]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[13]  Nicolas E. Gold,et al.  Lessons Learned in Exploring the Leap Motion(TM) Sensor for Gesture-based Instrument Design , 2014, NIME.

[14]  Peter Kazanzides,et al.  Comprehensive Tracker Based Display Calibration for Holographic Optical See-Through Head-Mounted Display , 2017, ArXiv.

[15]  Kenton O'Hara,et al.  Voice or Gesture in the Operating Room , 2015, CHI Extended Abstracts.

[16]  Yongtian Wang,et al.  A Modular Calibration Framework for 3D Interaction System Based on Optical See-Through Head-Mounted Displays in Augmented Reality , 2016, 2016 International Conference on Virtual Reality and Visualization (ICVRV).

[17]  Kenneth R. Moser,et al.  Evaluation of user-centric optical see-through head-mounted display calibration using a leap motion controller , 2016, 2016 IEEE Symposium on 3D User Interfaces (3DUI).

[18]  Spyros Vosinakis,et al.  Evaluation of visual feedback techniques for virtual grasping with bare hands using Leap Motion and Oculus Rift , 2018, Virtual Reality.

[19]  Tao Jiang,et al.  A New Operator Extracting Image Patch Based on EPLL , 2018, J. Inf. Process. Syst..

[20]  Sung-Ho Kim,et al.  Fire Detection Using Multi-Channel Information and Gray Level Co-occurrence Matrix Image Features , 2017, J. Inf. Process. Syst..

[21]  Chenfanfu Jiang,et al.  A virtual reality platform for dynamic human-scene interaction , 2016, SIGGRAPH ASIA Virtual Reality meets Physical Reality.

[22]  Woontack Woo,et al.  Bare Hand Interface for Interaction in the Video See-Through HMD Based Wearable AR Environment , 2006, ICEC.

[23]  Jack Sklansky,et al.  The Detection and Segmentation of Blobs in Infrared Images , 1981, IEEE Transactions on Systems, Man, and Cybernetics.