Grasp Recognition and Manipulation with the Tango

We describe a novel user interface for natural, whole hand interaction with 3D environments. Our interface uses a graspable device called the Tango, which looks like a ball but measures contact pressures on its surface at 256 tactual elements (taxels) at a high rate (100 Hz). The acceleration of the device is also measured. The key idea is to use this information to recognize the shape and movement of the user’s hand grasping the object. This allows the user to interact with 3D virtual objects using a hand avatar. The interface provides passive force feedback, and is easier to use than interfaces that require wearing gloves or other sensors on the hand. We describe a rotationally invariant matching algorithm for recognizing the hand shape from examples of previous interaction collected with motion capture. We also describe examples of 3D interaction using our system.

[1]  Dinesh K. Pai,et al.  Interaction capture and synthesis , 2005, SIGGRAPH 2005.

[2]  J. F. Soechting,et al.  Postural Hand Synergies for Tool Use , 1998, The Journal of Neuroscience.

[3]  Dimitris N. Metaxas,et al.  Model-based integration of visual cues for hand tracking , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[4]  Sean Quinlan,et al.  Efficient distance computation between non-convex objects , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[5]  Dinesh K. Pai,et al.  FootSee: an interactive animation system , 2003, SCA '03.

[6]  Shumin Zhai,et al.  The influence of muscle groups on performance of multiple degree-of-freedom input , 1996, CHI.

[7]  Katsushi Ikeuchi,et al.  A sensor fusion approach for recognizing continuous human grasping sequences using hidden Markov models , 2005, IEEE Transactions on Robotics.

[8]  Mary C. Whitton,et al.  Passive haptics significantly enhances virtual environments , 2001 .

[9]  Dinesh K. Pai,et al.  Programming contact tasks using a reality-based virtual environment integrated with vision , 1999, IEEE Trans. Robotics Autom..

[10]  Grigore C. Burdea,et al.  The Rutgers Master II-new design force-feedback glove , 2002 .

[11]  Nando de Freitas,et al.  Sequential Monte Carlo in Practice , 2001 .

[12]  Dinesh K. Pai,et al.  The Tango: a tangible tangoreceptive whole-hand human interface , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[13]  Dinesh K. Pai,et al.  Interaction capture and synthesis , 2005, ACM Trans. Graph..

[14]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.