Virtual Assembly/Disassembly System Using Natural Human Interaction and Control

The purpose of this research is to develop new a virtual assembly (VA) and disassembly system that uses natural interaction and control of objects and machinery. This VA system deals with parts, tools, machines and workplaces positioned in their usual places and working in the expected ways. We developed a system optimized for both performance and ease of use. For example, tools and machines work are controlled by a user with his/her hands, and he/she responds, moves, and works as expected. Instead of the traditional interactions with a keyboard/mouse, hand gestures, body movements, touching, holding, and carrying of the parts/tools/sub-assemblies, pushing buttons for machines and equipment controls are performed in a natural way. In addition, helpful information to enhance the immersiveness to substitute for missing natural factors like weight, inertia, and force are introduced. Depending on the desired effect, this may be the sound of a working machine, a rotating gear, a snapping sound between two parts, or the dropping a part on the floor or table. Further, a warning sound is used for: signalling of impossible operations, changing colour of object is used when hand/part collision detection occurs to show when a certain part can be grasped or assembled/disassembled. Finally, models and simulation optimization allow a high rendering frame rate and real-time interaction using moderate performance personal computers with affordable peripheral devices.