Learning gestures for interacting with low-fidelity prototypes

This paper presents an approach to help designers create their own application-specific gestures and evaluate them in user-studies based on low fidelity prototypes of the application they are designing. In order to learn custom gestures, we developed a machine learning tool that uses an anti-unification algorithm to learn based on samples of the gesture provided by the designer.

[1]  Teddy Seyed,et al.  Low-fidelity prototyping of gesture-based applications , 2011, EICS '11.

[2]  F BobickAaron,et al.  A State-Based Approach to the Representation and Recognition of Gesture , 1997 .

[3]  Mike Wu,et al.  Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays , 2003, UIST '03.

[4]  Jochen Burghardt,et al.  E-generalization using grammars , 2005, Artif. Intell..

[5]  Meredith Ringel Morris,et al.  ShadowGuides: visualizations for in-situ learning of multi-touch and whole-hand gestures , 2009, ITS '09.

[6]  Songyang Lao,et al.  A gestural interaction design model for multi-touch displays , 2009, BCS HCI.

[7]  Egor V. Kostylev,et al.  Anti-unification Algorithms and Their Applications in Program Analysis , 2009, Ershov Memorial Conference.

[8]  David Geerts,et al.  Low-Fidelity Prototyping for Multi-Touch Surfaces , 2010 .

[9]  Erica L. Olmsted-Hawala,et al.  The use of paper-prototyping in a low-fidelity usability study , 2009, 2009 IEEE International Professional Communication Conference.

[10]  mc schraefel,et al.  A Taxonomy of Gestures in Human Computer Interactions , 2005 .

[11]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[12]  M. Sheelagh T. Carpendale,et al.  Gestures in the wild: studying multi-touch gesture sequences on interactive tabletop exhibits , 2011, CHI.

[13]  Frank Maurer,et al.  A domain specific language to define gestures for multi-touch applications , 2010, DSM '10.

[14]  Meredith Ringel Morris,et al.  Understanding users' preferences for surface gestures , 2010, Graphics Interface.

[15]  Meredith Ringel Morris,et al.  User-defined gestures for surface computing , 2009, CHI.

[16]  Frank Maurer,et al.  Tool support for testing complex multi-touch gestures , 2010, ITS '10.

[17]  Robert J. Walker,et al.  Determining detailed structural correspondence for generalization tasks , 2007, ESEC-FSE '07.

[18]  Aaron F. Bobick,et al.  A State-Based Approach to the Representation and Recognition of Gesture , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Tracy Anne Hammond,et al.  Gesture Recognition Based on Manifold Learning , 2008, SSPR/SPR.

[20]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..