Physics-based Interactive Virtual Grasping

In this paper, we propose an interactive virtual grasping system, enabling physically-realistic interaction between a user and virtual objects, using kinematic hand models whose motions are tracked and controlled in real-time by the user. Since hand tracking devices provide only one-way coupling between the user and kinematic hands resulting in penetrations and friction-less interaction, our method implements the contacts as Coulomb's friction model and includes a dynamic proxy hand in the system which follows the kinematic virtual hand to provide a two-way control loop for the physics interactions. Our system, taking advantage of the graphics and physics engine of Unity3D, presently allows us to interact with 3D objects in the virtual space in real-time with no visible latency. Moreover, to visually assist the user's grasping we render the hand model as transparent and provide visual markers at the contact locations. We demonstrate how the use of these tools helps us to perform real-time grasping tasks in complicated virtual environment.