Simulating grasping behavior on an imaging interactive surface

We present techniques and algorithms to simulate grasping behavior on an imaging interactive surface (e.g., Microsoft Surface). In particular, we describe a contour model of touch contact shape, and show how these contours may be represented in a real-time physics simulation in a way that allows more realistic grasping behavior. For example, a virtual object may be moved by "squeezing" it with multiple contacts undergoing motion. The virtual object is caused to move by simulated contact and friction forces. Previous work [14] uses many small rigid bodies ("particle proxies") to approximate touch contact shape. This paper presents a variation of the particle proxy approach which allows grasping behavior. The advantages and disadvantages of this new approach are discussed.

[1]  Andrew D. Wilson.  PlayAnywhere: a compact interactive tabletop projection-vision system , 2005, UIST.

[2]  Andrew D. Wilson.  TouchLight: an imaging touch screen and display for gesture-based interaction , 2004, ICMI '04.

[3]  Andreas Butz,et al.  Interactions in the air: adding further depth to interactive tabletops , 2009, UIST '09.

[4]  Henrik Tramberend,et al.  Physically-based manipulation on the Responsive Workbench , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[5]  Chun-Jen Chen,et al.  A linear-time component-labeling algorithm using contour tracing technique , 2004, Comput. Vis. Image Underst..

[6]  Xiang Cao,et al.  ShapeTouch: Leveraging contact shape on interactive surfaces , 2008, 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems.

[7]  A.D. Wilson.  Depth-Sensing Video Cameras for 3D Tangible Tabletop Interaction , 2007, Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP'07).

[8]  Andrew D. Wilson.  Robust computer vision-based detection of pinching for one and two-handed gesture input , 2006, UIST.

[9]  Jefferson Y. Han.  Low-cost multi-touch sensing through frustrated total internal reflection , 2005, UIST.

[10]  Otmar Hilliges,et al.  Bringing physics to the surface , 2008, UIST '08.

[11]  Christine L. MacKenzie,et al.  The Grasping Hand , 2011, The Grasping Hand.

[12]  M. Sheelagh T. Carpendale,et al.  Fluid integration of rotation and translation , 2005, CHI.