The use of visual and auditory feedback for assembly task performance in a virtual environment

This paper presents our creation and evaluation of multi-modal interface for a virtual assembly environment. It involves implementing an assembly simulation environment with multi-sensory feedback (visual and auditory), and evaluating the effects of multi-modal feedback on assembly task performance. This virtual environment experimental platform brought together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. A peg-in-a-hole and a Sener electronic box assembly tasks have been used as the task cases to perform human factor experiments, using sixteen subjects. Both objective performance data (task completion time, and human performance error rates) and subjective opinions (questionnaires) have been gathered from this experiment.

[1]  Ying Zhang,et al.  Integration of 3D Sound Feedback into a Virtual Assembly Environment , 2003 .

[2]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[3]  Terrence Fernando,et al.  A Distributed Virtual Environment for Collaborative Engineering , 1998, Presence.

[4]  Ming C. Lin,et al.  Collision Detection between Geometric Models: A Survey , 1998 .

[5]  Dinesh K. Pai,et al.  FoleyAutomatic: physically-based sound effects for interactive simulation and animation , 2001, SIGGRAPH.

[6]  Dinesh K. Pai,et al.  Physically-based Sound Eects for Interactive Simulation and Animation , 2001 .

[7]  Torsten Kuhlen,et al.  MAESTRO - a tool for interactive assembly simulation in virtual environments , 2001, EGVE/IPT.

[8]  James F. O'Brien,et al.  Synthesizing sounds from physically based motion , 2001, SIGGRAPH.

[9]  Terrence Fernando,et al.  A constraint manager to support virtual maintainability , 2003, Comput. Graph..

[10]  Constantine Stephanidis,et al.  Human-Computer Interaction : Theory and Practice (part 1), Volume 1 , 2005 .

[11]  James K. Hahn,et al.  Integrating Sounds and Motions in Virtual Environments , 1998, Presence.

[12]  Fan Dai,et al.  Virtual Reality for Industrial Applications , 1998, Computer Graphics: Systems and Applications.

[13]  Yoshifumi Kitamura,et al.  A Sophisticated Manipulation Aid in a Virtual Environment using Dynamic Constraints among Object Faces , 1998, Presence.

[14]  Yong Wang,et al.  VADE: A Virtual Assembly Design Environment , 1999, IEEE Computer Graphics and Applications.

[15]  Ying Zhang,et al.  Evaluation of auditory feedback on task performance in virtual assembly environment , 2004, The Fourth International Conference onComputer and Information Technology, 2004. CIT '04..

[16]  Ying Zhang,et al.  3D auditory feedback acts as task aid in a virtual assembly environment , 2003, Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium (Cat. No.03EX694).

[17]  James F. O'Brien,et al.  Synthesizing Sounds from Physically Based Motion , 2001, SIGGRAPH Video Review on Animation Theater Program.

[18]  Elizabeth M. Wenzel,et al.  Localization in Virtual Acoustic Displays , 1992, Presence: Teleoperators & Virtual Environments.