Evaluation of Auditory and Visual Feedback on Task Performance in a Virtual Assembly Environment

This paper presents the creation of an assembly simulation environment with multisensory feedback (auditory and visual), and the evaluation of the effects of auditory and visual feedback on the task performance in the context of assembly simulation in a virtual environment (VE). This VE experimental system platform brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real time 3D sound generation technology around a virtual reality workbench and a common software platform. A peg-in-a-hole and a Sener electronic box assembly task have been used as the task cases to conduct the human factor experiment, using sixteen participants. Both objective performance data (i.e., task completion time, TCT; and human performance error rate, HPER) and subjective opinions (i.e., questionnaires) on the utilization of auditory and visual feedback in a virtual assembly environment (VAE) have been gathered from the experiment. Results showed that the introduction of auditory and/or visual feedback into VAE did improve the assembly task performance. They also indicated that integrated feedback (auditory plus visual) offered better assembly task performance than either feedback used in isolation. Most participants preferred integrated feedback to either individual feedback (auditory or visual) or no feedback. The participants' comments demonstrated that nonrealistic or inappropriate feedback had a negative effect on the task performance, and easily made them frustrated.

[1]  James K. Hahn,et al.  Integrating Sounds in Virtual Environments. , 1998 .

[2]  Ying Zhang,et al.  Integration of 3D Sound Feedback into a Virtual Assembly Environment , 2003 .

[3]  Dan Zetu,et al.  Virtual Manufacturing , 2001 .

[4]  Terrence Fernando,et al.  A Distributed Virtual Environment for Collaborative Engineering , 1998, Presence.

[5]  Torsten Kuhlen,et al.  MAESTRO - a tool for interactive assembly simulation in virtual environments , 2001, EGVE/IPT.

[6]  Yong Wang,et al.  VADE: A Virtual Assembly Design Environment , 1999, IEEE Computer Graphics and Applications.

[7]  Ying Zhang,et al.  3D sound feedback act as task aid in a virtual assembly environment , 2003, Proceedings of Theory and Practice of Computer Graphics, 2003..

[8]  Andrew Y. C. Nee,et al.  Virtual Reality and Augmented Reality Applications in Manufacturing , 2004 .

[9]  Fan Dai,et al.  Virtual Reality for Industrial Applications , 1998, Computer Graphics: Systems and Applications.

[10]  Dinesh K. Pai,et al.  FoleyAutomatic: physically-based sound effects for interactive simulation and animation , 2001, SIGGRAPH.

[11]  Terrence Fernando,et al.  A constraint manager to support virtual maintainability , 2003, Comput. Graph..

[12]  Ying Zhang,et al.  The use of visual and auditory feedback for assembly task performance in a virtual environment , 2005, Ninth International Conference on Information Visualisation (IV'05).

[13]  Ming C. Lin,et al.  Collision Detection between Geometric Models: A Survey , 1998 .

[14]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[15]  Yoshifumi Kitamura,et al.  A Sophisticated Manipulation Aid in a Virtual Environment using Dynamic Constraints among Object Faces , 1998, Presence.

[16]  James F. O'Brien,et al.  Synthesizing sounds from physically based motion , 2001, SIGGRAPH.

[17]  Elizabeth M. Wenzel,et al.  Localization in Virtual Acoustic Displays , 1992, Presence: Teleoperators & Virtual Environments.