Augmented reality for assembly guidance using a virtual interactive tool

The application of augmented reality (AR) technology for assembly guidance is a novel approach in the traditional manufacturing domain. In this paper, we propose an AR approach for assembly guidance using a virtual interactive tool that is intuitive and easy to use. The virtual interactive tool, termed the virtual interaction panel (VirIP), is an easy-to-use tool that can be used to interactively control AR systems. The VirIP is composed of virtual buttons, which have meaningful assembly information that can be activated by an interaction pen during the assembly process. The interaction pen can be any general pen-like object with a certain colour distribution. It is tracked using a restricted coulomb energy (RCE) network in real-time and used to trigger the relevant buttons in the VirIPs for assembly guidance. Meanwhile, a visual assembly tree structure (VATS) is used for information management and assembly instructions retrieval in this AR environment. VATS is a hierarchical tree structure that can be easily maintained via a visual interface. It can be directly integrated into the AR system or it can alternatively act as an independent central control station on a remote computer to control the data flow of the assembly information. This paper describes a typical scenario for assembly guidance using VirIP and VATS. The main characteristic of the proposed AR system is the intuitive way in which an assembly operator can easily step through a pre-defined assembly plan/sequence without the need of any sensor schemes or markers attached on the assembly components. Several experiments were conducted to validate the performance of the proposed AR-based method using a monitor and a head-mounted display. The results show that the AR-based method can provide an efficient way for assembly guidance.

[1]  Rajeev Sharma,et al.  Interactive evaluation of assembly sequences using augmented reality , 1999, IEEE Trans. Robotics Autom..

[2]  Gunther Reinhart,et al.  Integrating Augmented Reality in the Assembly Domain - Fundamentals, Benefits and Applications , 2003 .

[3]  Dave Sims,et al.  New realities in aircraft design and manufacture , 1994, IEEE Computer Graphics and Applications.

[4]  Frank Biocca,et al.  Comparative effectiveness of augmented reality in object assembly , 2003, CHI '03.

[5]  Didier Stricker,et al.  Augmented reality for construction tasks: doorlock assembly , 1999 .

[6]  D. L. Reilly,et al.  A neural model for category learning , 1982, Biological Cybernetics.

[7]  Steven Hogg,et al.  Wearable augmented virtual reality for enhancing information delivery in high precision defence assembly: an engineering case study , 2004, Virtual Reality.

[8]  Werner Hartmann,et al.  Authoring of a mixed reality furniture assembly instructor , 2003, SIGGRAPH '03.

[9]  Olaf Oehme,et al.  Augmented Reality (AR) for Assembly Processes Design and Experimental Evaluation , 2003, Int. J. Hum. Comput. Interact..

[10]  Werner Hartmann,et al.  Authoring of a mixed reality assembly instructor for hierarchical structures , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[11]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[12]  Ming Xie,et al.  Hand image segmentation using color and RCE neural network , 2001, Robotics Auton. Syst..

[13]  Adam L. Janin,et al.  Several devils in the details: making an AR application work in the airplane factory , 1999 .

[14]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[15]  Woodrow Barfield,et al.  Evaluating the effectiveness of augmented reality displays for a manual assembly task , 1999, Virtual Reality.

[16]  Alfredo Liverani,et al.  A CAD-augmented Reality Integrated Environment for Assembly Sequence Check and Interactive Validation , 2004, Concurr. Eng. Res. Appl..

[17]  Ronald Azuma,et al.  Evaluating label placement for augmented reality view management , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[18]  Rajeev Sharma,et al.  Computer vision and augmented reality for guiding assembly , 2002 .

[19]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.