Assembly Design and Evaluation Based on Bare-Hand Interaction in an Augmented Reality Environment

Augmented Reality (AR) is an increasingly important application domain for computer graphics and user interface design. In this paper, the authors propose an AR interface for assembly design and evaluation. The methodologies to establish an AR assembly environment and the architecture of the proposed system are presented. A key novelty of this research is the use of bare hands for human-computer interaction in the AR assembly environment. An effective hand segmentation method based on a Restricted Coulomb Energy (RCE) neural network is developed. Experimental results show that the proposed hand segmentation method is effective and robust and the finger tracking algorithm runs in real time under a wide range of lighting conditions. An experiment using two fingertips to control virtual objects to simulate assembly in a 2D space is conducted.

[1]  Hsin-Yi Lai,et al.  A systematic approach for automatic assembly sequence plan generation , 2004 .

[2]  Raffaele de Amicis,et al.  Spacedesign: A Mixed Reality Workspace for Aesthetic Industrial Design , 2002, ISMAR.

[3]  Werner Hartmann,et al.  Authoring of a mixed reality assembly instructor for hierarchical structures , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[4]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[5]  James M. Kasson,et al.  An analysis of selected computer interchange color spaces , 1992, TOGS.

[6]  Olaf Oehme,et al.  Augmented Reality (AR) for Assembly Processes Design and Experimental Evaluation , 2003, Int. J. Hum. Comput. Interact..

[7]  Soh-Khim Ong,et al.  Markerless Augmented Reality Using a Robust Point Transferring Method , 2007, MMM.

[8]  Hayit Greenspan,et al.  Mixture model for face-color modeling and segmentation , 2001, Pattern Recognit. Lett..

[9]  Ming Xie,et al.  Hand image segmentation using color and RCE neural network , 2001, Robotics Auton. Syst..

[10]  Woodrow Barfield,et al.  Fundamentals of Wearable Computers and Augumented Reality , 2000 .

[11]  F Zorriassatine,et al.  A survey of virtual prototyping techniques for mechanical product development , 2003 .

[12]  Bernd Menser,et al.  Segmentation and tracking of facial regions in color image sequences , 2000, Visual Communications and Image Processing.

[13]  Andrew Y. C. Nee,et al.  Augmented reality for assembly guidance using a virtual interactive tool , 2008 .

[14]  Yakup Genc,et al.  Marker-less tracking for AR: a learning-based approach , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[15]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[16]  Bruce H. Thomas,et al.  Glove Based User Interaction Techniques for Augmented Reality in an Outdoor Environment , 2002, Virtual Reality.

[17]  Frank Biocca,et al.  Comparative effectiveness of augmented reality in object assembly , 2003, CHI '03.

[18]  Pascal Fua,et al.  A framework for rapid evaluation of prototypes with augmented reality , 2000, VRST '00.

[19]  Didier Stricker,et al.  Augmented reality for construction tasks: doorlock assembly , 1999 .

[20]  Tuomo Tuikka,et al.  Augmenting virtual prototyping with physical objects , 2000, AVI '00.

[21]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[22]  A. Bouzerdoum,et al.  A Bayesian approach to skin color classification in YCbCr color space , 2000, 2000 TENCON Proceedings. Intelligent Systems and Technologies for the New Millennium (Cat. No.00CH37119).