A Unified Framework for Augmented Reality and Knowledge-Based Systems in Maintaining Aircraft

Aircraft maintenance and training play one of the most important roles in ensuring flight safety. The maintenance process usually involves massive numbers of components and substantial procedural knowledge of maintenance procedures. Maintenance tasks require technicians to follow rigorous procedures to prevent operational errors in the maintenance process. In addition, the maintenance time is a cost-sensitive issue for airlines. This paper proposes intelligent augmented reality (IAR) system to minimize operation errors and time-related costs and help aircraft technicians cope with complex tasks by using an intuitive UI/UX interface for their maintenance tasks. The IAR system is composed mainly of three major modules: 1) the AR module 2) the knowledge-based system (KBS) module 3) a unified platform with an integrated UI/UX module between the AR and KBS modules. The AR module addresses vision-based tracking, annotation, and recognition. The KBS module deals with ontology-based resources and context management. Overall testing of the IAR system is conducted at Korea Air Lines (KAL) hangars. Tasks involving the removal and installation of pitch trimmers in landing gear are selected for benchmarking purposes, and according to the results, the proposed IAR system can help technicians to be more effective and accurate in performing their maintenance tasks.

[1]  Luigi di Stefano,et al.  Augmented Reality for Aircraft Maintenance Training and Operations Support , 2011, IEEE Computer Graphics and Applications.

[2]  Stephen Fox,et al.  The importance of information and communication design for manual skills instruction with augmented reality , 2010 .

[3]  Ulrich Neumann,et al.  The Gixel array descriptor (GAD) for multimodal image matching , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[4]  Quan Wang,et al.  Augmented distinctive features for efficient image matching , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[5]  James Gao,et al.  A Web-based Product Service System for aerospace maintenance, repair and overhaul services , 2012, Comput. Ind..

[6]  Ahmad Nurzid Rosli,et al.  Ontology-driven visualization system for semantic searching , 2011, Multimedia Tools and Applications.

[7]  Andrew Y. C. Nee,et al.  An authorable context-aware augmented reality system to assist the maintenance technicians , 2012 .

[8]  Steven K. Feiner,et al.  Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair , 2011, IEEE Transactions on Visualization and Computer Graphics.

[9]  Frank Boukamp,et al.  Ontology-Based Approach to Context Representation and Reasoning for Managing Context-Sensitive Construction Information , 2011, J. Comput. Civ. Eng..

[10]  Jing Huang,et al.  Multimodal image matching using self similarity , 2011, 2011 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[11]  Thomas B. Moeslund,et al.  IEEE Workshop on Applications of Computer Vision, WACV 2005 , 2005 .