Combining visual natural markers and IMU for improved AR based indoor navigation

The operation and maintenance phase is the longest and most expensive life-cycle period of buildings and facilities. Operators need to carry out activities to maintain equipment to prevent functionality failures. Although some software tools have already been introduced, research studies have concluded that (1) facility handover data is still predominantly dispersed, unformatted and paper-based and (2) hence operators still spend 50% of their on-site work on target localization and navigation. To improve these procedures, the authors previously presented a natural marker-based Augmented Reality (AR) framework that digitally supports facility maintenance operators when navigating indoors. Although previous results showed the practical potential, this framework fails if no visual marker is available, if identical markers are at multiple locations, and if markers are light emitting signs. To overcome these shortcomings, this paper presents an improved method that combines an Inertial Measurement Unit (IMU) based step counter and visual live video feed for AR based indoor navigation support. In addition, the AR based marker detection procedure is improved by learning camera exposure times in case of light emitting markers. A case study and experimental results in a controlled environment reveal the improvements and advantages of the enhanced framework.

[1]  Markus König,et al.  Natural markers for augmented reality-based indoor navigation and facility maintenance , 2014 .

[2]  I-Tung Yang,et al.  An invisible head marker tracking system for indoor mobile augmented reality , 2013 .

[3]  Shuozhi Liu,et al.  Shortest Path Optimization of the Dijkstra Algorithm on the Rail Transit Cohesion Planning , 2013 .

[4]  Uwe Rüppel,et al.  BIM-based immersive indoor graph networks for emergency situations in buildings , 2010 .

[5]  Ömer Akin,et al.  Augmented reality-based computational fieldwork support for equipment operations and maintenance , 2011 .

[6]  Florian Gschwandtner,et al.  BIGML: A Location Model with Individual Waypoint Graphs for Indoor Location-based Services , 2010, PIK Prax. Informationsverarbeitung Kommun..

[7]  Peter Martini,et al.  Indoor tracking for mission critical scenarios: A survey , 2011, Pervasive Mob. Comput..

[8]  Oh-Seong Kwon,et al.  A framework for proactive construction defect management using BIM, augmented reality and ontology-based data collection template , 2013 .

[9]  Mani Golparvar-Fard,et al.  High-precision vision-based mobile augmented reality system for context-aware architectural, engineering, construction and facility management (AEC/FM) applications , 2013 .

[10]  Shun Cao,et al.  A Pedestrian Dead Reckoning System Integrating Low-Cost MEMS Inertial Sensors and GPS Receiver , 2014 .

[11]  Joachim Hertzberg,et al.  An autonomous mobile robot with a 3D laser range finder for 3D exploration and digitalization of indoor environments , 2003, Robotics Auton. Syst..

[12]  Jorge A. Baier,et al.  Making A* Run Faster than D*-Lite for Path-Planning in Partially Known Terrain , 2014, ICAPS.

[13]  Shuji Hashimoto,et al.  An intelligent localization algorithm using read time of RFID system , 2010, Adv. Eng. Informatics.

[14]  Andreas Nüchter,et al.  A mobile robot based system for fully automated thermal 3D mapping , 2014, Adv. Eng. Informatics.

[15]  Quentin Ladetto,et al.  On foot navigation: continuous step calibration using both complementary recursive prediction and adaptive Kalman filtering , 2000 .

[16]  Ulrich Walder,et al.  Context-adaptive algorithms to improve indoor positioning with inertial sensors , 2010, 2010 International Conference on Indoor Positioning and Indoor Navigation.

[17]  Markus König,et al.  Performance Study on Natural Marker Detection for Augmented Reality Supported Facility Maintenance , 2013 .

[18]  Amin Hammad,et al.  Localization of RFID-equipped assets during the operation phase of facilities , 2013, Adv. Eng. Informatics.

[19]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[20]  Marc Pollefeys,et al.  Vision-based autonomous mapping and exploration using a quadrotor MAV , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Robert Harle,et al.  Pedestrian localisation for indoor environments , 2008, UbiComp.

[22]  Burcin Becerik-Gerber,et al.  Performance-based evaluation of RFID-based indoor location sensing solutions for the built environment , 2011, Adv. Eng. Informatics.

[23]  Yasushi Makihara,et al.  Inertial-sensor-based walking action recognition using robust step detection and inter-class relationships , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[24]  Vineet R. Kamat,et al.  High-precision identification of contextual information in location-aware engineering applications , 2009, Adv. Eng. Informatics.

[25]  Supun Samarasekera,et al.  Multi-sensor navigation algorithm using monocular camera, IMU and GPS for large scale augmented reality , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[26]  E W East,et al.  Construction Operations Building Information Exchange (COBIE): Requirements Definition and Pilot Implementation Standard , 2007 .

[27]  Osama Moselhi,et al.  GPS-less indoor construction location sensing , 2012 .

[28]  Vineet R. Kamat,et al.  Evaluation of position tracking technologies for user localization in indoor construction environments , 2009 .

[29]  Neil Zhao.  Full-Featured Pedometer Design Realized with 3-Axis Digital Accelerometer , 2010 .

[30]  Thomas Liebich,et al.  Facility Management Handover Model View , 2013, J. Comput. Civ. Eng..

[31]  Osama Moselhi,et al.  RFID indoor location identification for construction projects , 2012 .