TouchTokens: Guiding Touch Patterns with Passive Tokens

TouchTokens make it possible to easily build interfaces that combine tangible and gestural input using passive tokens and a regular multi-touch surface. The tokens constrain users' grasp, and thus, the relative spatial configuration of fingers on the surface, theoretically making it possible to design algorithms that can recognize the resulting touch patterns. We performed a formative user study to collect and analyze touch patterns with tokens of varying shape and size. The analysis of this pattern collection showed that individual users have a consistent grasp for each token, but that this grasp is user-dependent and that different grasp strategies can lead to confounding patterns. We thus designed a second set of tokens featuring notches that constrain users' grasp. Our recognition algorithm can classify the resulting patterns with a high level of accuracy (>95%) without any training, enabling application designers to associate rich touch input vocabularies with command triggers and parameter controls.

[1]  Hiroshi Ishii,et al.  Bricks: laying the foundations for graspable user interfaces , 1995, CHI '95.

[2]  Ricardo Langner,et al.  Grids & guides: multi-touch layout and alignment tools , 2011, CHI.

[3]  Patrick Baudisch,et al.  Fiberio: a touchscreen that senses fingerprints , 2013, UIST.

[4]  Shumin Zhai,et al.  Foundational Issues in Touch-Surface Stroke Gesture Design - An Integrative Review , 2012, Found. Trends Hum. Comput. Interact..

[5]  Daniel Vogel,et al.  Conté: multimodal input inspired by an artist's crayon , 2011, UIST.

[6]  Yuichi Itoh,et al.  PUCs: detecting transparent, passive untouched capacitive widgets on unmodified multi-touch displays , 2013, ITS.

[7]  Tony DeRose,et al.  Toolglass and magic lenses: the see-through interface , 1993, SIGGRAPH.

[8]  Caroline Appert,et al.  Prospective motor control on tabletops: planning grasp for multitouch interaction , 2014, CHI.

[9]  Michael Rohs,et al.  CapWidgets: tangile widgets versus multi-touch controls on mobile devices , 2011, CHI Extended Abstracts.

[10]  B. Buchholz,et al.  Anthropometric data for describing the kinematics of the human hand. , 1992, Ergonomics.

[11]  Ali Mazalek,et al.  Exploring the design space of gestural interaction with active tokens through user-defined gestures , 2014, CHI.

[12]  Patrick Baudisch,et al.  Lumino: tangible blocks for tabletop computers based on glass fiber bundles , 2010, CHI.

[13]  Xiang Cao,et al.  ShapeTouch: Leveraging contact shape on interactive surfaces , 2008, 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems.

[14]  Beryl Plimmer,et al.  CapTUI: Geometric Drawing with Tangibles on a Capacitive Multi-touch Display , 2013, INTERACT.

[15]  Olivier Bau,et al.  Arpège: learning multitouch chord gestures vocabularies , 2013, ITS.

[16]  Ian Oakley,et al.  Designing tangible magnetic appcessories , 2013, TEI '13.

[17]  Ross Bencina,et al.  reacTIVision: a computer-vision framework for table-based tangible interaction , 2007, TEI.

[18]  Desney S. Tan,et al.  Enhancing input on and above the interactive surface with muscle sensing , 2009, ITS '09.

[19]  Radu-Daniel Vatavu,et al.  Automatic recognition of object size and shape via user-dependent measurements of the grasping hand , 2013, Int. J. Hum. Comput. Stud..

[20]  J. F. Soechting,et al.  Gradual molding of the hand to object contours. , 1998, Journal of neurophysiology.

[21]  Natasa Milic-Frayling,et al.  Materializing the query with facet-streams: a hybrid surface for collaborative search on tabletops , 2011, CHI.

[22]  Hideki Koike,et al.  Transparent 2-D markers on an LCD tabletop system , 2009, CHI.

[23]  Sergi Jordà,et al.  The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces , 2007, TEI.

[24]  Raimund Dachselt,et al.  T4 - transparent and translucent tangibles on tabletops , 2014, AVI.

[25]  Caroline Appert,et al.  Multi-touch gestures for discrete and continuous control , 2014, AVI.

[26]  S. Lederman,et al.  TACTUAL PERCEPTION , 2003 .

[27]  Björn Hartmann,et al.  Augmenting interactive tables with mice & keyboards , 2009, UIST '09.

[28]  Robert Xiao,et al.  TouchTools: leveraging familiarity and skill with physical tools to augment touch interaction , 2014, CHI.

[29]  Hiroshi Ishii,et al.  Tangible Query Interfaces: Physically Constrained Tokens for Manipulating Database Queries , 2003, INTERACT.

[30]  Andreas Butz,et al.  Sketch-a-TUI: low cost prototyping of tangible interactions using cardboard and conductive ink , 2012, Tangible and Embedded Interaction.

[31]  Stefanie Müller,et al.  CapStones and ZebraWidgets: sensing stacks of building blocks, dials and sliders on capacitive touch screens , 2012, CHI.

[32]  Mike Wu,et al.  Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays , 2003, UIST '03.

[33]  Lars Erik Holmquist,et al.  Token-Based Acces to Digital Information , 1999, HUC.

[34]  Tracy Anne Hammond,et al.  Object interaction detection using hand posture cues in an office setting , 2011, Int. J. Hum. Comput. Stud..

[35]  Jörg Müller,et al.  Design and evaluation of finger-count interaction: Combining multitouch gestures and menus , 2012, Int. J. Hum. Comput. Stud..

[36]  Pierre Dragicevic,et al.  Tangible remote controllers for wall-size displays , 2012, CHI.

[37]  Jun Rekimoto,et al.  DataTiles: a modular platform for mixed physical and graphical interactions , 2001, CHI.

[38]  Jun Rekimoto,et al.  SmartSkin: an infrastructure for freehand manipulation on interactive surfaces , 2002, CHI.

[39]  Jeffrey Heer,et al.  SpanningAspectRatioBank Easing FunctionS ArrayIn ColorIn Date Interpolator MatrixInterpola NumObjecPointI Rectang ISchedu Parallel Pause Scheduler Sequen Transition Transitioner Transiti Tween Co DelimGraphMLCon IData JSONCon DataField DataSc Dat DataSource Data DataUtil DirtySprite LineS RectSprite , 2011 .

[40]  Atsushi Sugiura,et al.  A user interface using fingerprint recognition: holding commands and data objects on fingers , 1998, UIST '98.

[41]  Li-Wei Chan,et al.  GaussStones: shielded magnetic tangibles for multi-token interactions on portable displays , 2014, UIST.

[42]  Elena Mugellini,et al.  Tangible Meets Gestural: Comparing and Blending Post-WIMP Interaction Paradigms , 2015, Tangible and Embedded Interaction.

[43]  Patrick Olivier,et al.  SurfaceMouse: supplementing multi-touch interaction with a virtual mouse , 2011, Tangible and Embedded Interaction.

[44]  Hiroshi Ishii,et al.  mediaBlocks: physical containers, transports, and controls for online media , 1998, SIGGRAPH.

[45]  Geehyuk Lee,et al.  Force gestures: augmenting touch screen gestures with normal and tangential forces , 2011, UIST.

[46]  Tom Bartindale,et al.  Stacks on the surface: resolving physical order using fiducial markers with structured transparency , 2009, ITS '09.

[47]  Patrick Baudisch,et al.  Precise selection techniques for multi-touch screens , 2006, CHI.

[48]  James D. Hollan,et al.  SLAP widgets: bridging the gap between virtual and physical controls on tabletops , 2009, CHI.

[49]  HeerJeffrey,et al.  D3 Data-Driven Documents , 2011 .

[50]  Li-Wei Chan,et al.  GaussBits: magnetic tangible bits for portable and occlusion-free near-surface interactions , 2013, CHI Extended Abstracts.

[51]  Mike Y. Chen,et al.  Clip-on gadgets: expanding multi-touch interaction area with unpowered tactile controls , 2011, UIST '11.

[52]  Mike Wu,et al.  Gesture registration, relaxation, and reuse for multi-point direct-touch surfaces , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[53]  Ian Oakley,et al.  MagnID: Tracking Multiple Magnetic Tokens , 2015, Tangible and Embedded Interaction.

[54]  Chris Harrison,et al.  TapSense: enhancing finger interaction on touch surfaces , 2011, UIST.

[55]  Xiang 'Anthony' Chen,et al.  The fat thumb: using the thumb's contact size for single-handed mobile interaction , 2012, Mobile HCI.

[56]  Meredith Ringel Morris,et al.  ShadowGuides: visualizations for in-situ learning of multi-touch and whole-hand gestures , 2009, ITS '09.

[57]  Daniel J. Wigdor,et al.  Rock & rails: extending multi-touch interactions with shape gestures to enable precise spatial manipulations , 2011, CHI.