Evaluation of HoloLens Tracking and Depth Sensing for Indoor Mapping Applications

The Microsoft HoloLens is a head-worn mobile augmented reality device that is capable of mapping its direct environment in real-time as triangle meshes and localize itself within these three-dimensional meshes simultaneously. The device is equipped with a variety of sensors including four tracking cameras and a time-of-flight (ToF) range camera. Sensor images and their poses estimated by the built-in tracking system can be accessed by the user. This makes the HoloLens potentially interesting as an indoor mapping device. In this paper, we introduce the different sensors of the device and evaluate the complete system in respect of the task of mapping indoor environments. The overall quality of such a system depends mainly on the quality of the depth sensor together with its associated pose derived from the tracking system. For this purpose, we first evaluate the performance of the HoloLens depth sensor and its tracking system separately. Finally, we evaluate the overall system regarding its capability for mapping multi-room environments.

[1]  Jens Lambrecht,et al.  Augmented-Reality-Based Visualization of Navigation Data of Mobile Robots on the Microsoft Hololens - Possibilities and Limitations , 2019, 2019 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM).

[2]  Kourosh Khoshelham,et al.  Indoor Mapping Eyewear: Geometric Evaluation of Spatial Mapping Capability of Hololens , 2019 .

[3]  Celeste M. Alfes,et al.  Virtually Nursing: Emerging Technologies in Nursing Education , 2017, Nurse educator.

[4]  Jan Guhl,et al.  Concept and architecture for programming industrial robots using augmented reality with mobile devices like microsoft HoloLens , 2017, 2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[5]  Martin Weinmann,et al.  Evaluation of the Microsoft HoloLens for the Mapping of Indoor Building Environments , 2019 .

[6]  Andrea Masiero,et al.  Performance Evaluation of Two Indoor Mapping Systems: Low-Cost UWB-Aided Photogrammetry and Backpack Laser Scanning , 2018 .

[7]  Markus Aleksy,et al.  Utilizing HoloLens to Support Industrial Service Processes , 2018, 2018 IEEE 32nd International Conference on Advanced Information Networking and Applications (AINA).

[8]  Boris Jutzi,et al.  SEMI-AUTOMATIC IMAGE-BASED CO-REGISTRATION OF RANGE IMAGING DATA WITH DIFFERENT CHARACTERISTICS , 2013 .

[9]  Berardo Naticchia,et al.  Augmented Reality Application Supporting On-Site Secondary Building Assets Management , 2019 .

[10]  KolbAndreas,et al.  Kinect range sensing , 2015 .

[11]  Henrique Lorenzo,et al.  Behavior Analysis of Novel Wearable Indoor Mapping System Based on 3D-SLAM , 2018, Sensors.

[12]  Yuwei Chen,et al.  The Accuracy Comparison of Three Simultaneous Localization and Mapping (SLAM)-Based Indoor Mapping Technologies † , 2018, Sensors.

[13]  Christian Riess,et al.  A Comparative Error Analysis of Current Time-of-Flight Sensors , 2016, IEEE Transactions on Computational Imaging.

[14]  Pedro Arias,et al.  Metrological comparison between Kinect I and Kinect II sensors , 2015 .

[15]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Jana Jost,et al.  Evaluation of the Application of Smart Glasses for Decentralized Control Systems in Logistics , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[17]  Andreas Kolb,et al.  Kinect range sensing: Structured-light versus Time-of-Flight Kinect , 2015, Comput. Vis. Image Underst..

[18]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[19]  Adam Rankin,et al.  Hologram stability evaluation for Microsoft HoloLens , 2017, Medical Imaging.

[20]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[22]  Juha Hyyppä,et al.  Comparison of the Selected State-Of-The-Art 3D Indoor Scanning and Point Cloud Generation Methods , 2017, Remote. Sens..

[23]  Hao Wang,et al.  Application of Mixed Reality Based on Hololens in Nuclear Power Engineering , 2019 .

[24]  George Vosselman,et al.  An Evaluation Pipeline for Indoor Laser Scanning Point Clouds , 2018 .

[25]  Fabio Remondino,et al.  Investigation of indoor and outdoor performance of two portable mobile mapping systems , 2017, Optical Metrology.

[26]  H. Macher,et al.  FIRST EXPERIENCES WITH KINECT V2 SENSOR FOR CLOSE RANGE 3D MODELLING , 2015 .

[27]  Stephan Nebiker,et al.  DEVELOPMENT OF A PORTABLE HIGH PERFORMANCE MOBILE MAPPING SYSTEM USING THE ROBOT OPERATING SYSTEM , 2018, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[28]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Abdulmotaleb El-Saddik,et al.  Technical Evaluation of HoloLens for Multimedia: A First Look , 2018, IEEE MultiMedia.

[30]  Wenbin Guo.  Improving Engineering Education Using Augmented Reality Environment , 2018, HCI.

[31]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[32]  Abdulmotaleb El-Saddik,et al.  Towards a QoE Model to Evaluate Holographic Augmented Reality Devices , 2019, IEEE MultiMedia.

[33]  知秋.  Microsoft：微软“变脸” , 2006 .