Tool support for testing complex multi-touch gestures

Though many tabletop applications allow users to interact with the application using complex multi-touch gestures, automated tool support for testing such gestures is limited. As a result, gesture-based interactions with an application are often tested manually, which is an expensive and error prone process. In this paper, we present TouchToolkit, a tool designed to help developers automate their testing of gestures by incorporating recorded gestures into unit tests. The design of TouchToolkit was informed by a small interview study conducted to explore the challenges software developers face when debugging and testing tabletop applications. We have also conducted a preliminary evaluation of the tool with encouraging results.

[1]  Frank Maurer,et al.  A domain specific language to define gestures for multi-touch applications , 2010, DSM '10.

[2]  Enrico Costanza,et al.  TUIO: A Protocol for Table-Top Tangible User Interfaces , 2005 .

[3]  Joyojeet Pal,et al.  Multiple Mice for Computers in Education in Developing Countries , 2006, 2006 International Conference on Information and Communication Technologies and Development.

[4]  Tracy L. Westeyn,et al.  Georgia tech gesture toolkit: supporting experiments in gesture recognition , 2003, ICMI '03.

[5]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[6]  Xiang Cao,et al.  Mouse 2.0: multi-touch meets the mouse , 2009, UIST '09.

[7]  Gudrun Klinker,et al.  A multitouch software architecture , 2008, NordiCHI.

[8]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[9]  Chris North,et al.  Understanding Multi-touch Manipulation for Surface Computing , 2009, INTERACT.

[10]  Jerry Alan Fails,et al.  A design tool for camera-based interaction , 2003, CHI '03.

[11]  James A. Landay,et al.  Quill: a gesture design tool for pen-based user interfaces , 2001 .

[12]  Kent Beck,et al.  Extreme Programming Explained: Embrace Change (2nd Edition) , 2004 .

[13]  P. Gray,et al.  A Demonstration of the OpenInterface Interaction Development Environment , 2007 .

[14]  Thad Starner,et al.  MAGIC: a motion gesture design tool , 2010, CHI.

[15]  Juan Pablo Hourcade,et al.  PyMT: a post-WIMP multi-touch user interface toolkit , 2009, ITS '09.

[16]  Levent Burak Kara,et al.  An image-based, trainable symbol recognizer for hand-drawn sketches , 2005, Comput. Graph..

[17]  Meredith Ringel Morris,et al.  User-defined gestures for surface computing , 2009, CHI.

[18]  Ian Witten,et al.  Data Mining , 2000 .

[19]  Johnny Chung Lee,et al.  Hacking the Nintendo Wii Remote , 2008, IEEE Pervasive Computing.