Interaction and recognition challenges in interpreting children's touch and gesture input on mobile devices

As mobile devices like the iPad and iPhone become increasingly commonplace, touchscreen interactions are quickly overtaking other interaction methods in terms of frequency and experience for many users. However, most of these devices have been designed for the general, typical user. Trends indicate that children are using these devices (either their parents' or their own) for entertainment or learning activities. Previous work has found key differences in how children use touch and surface gesture interaction modalities vs. adults. In this paper, we specifically examine the impact of these differences in terms of automatically and reliably understanding what kids meant to do. We present a study of children and adults performing touch and surface gesture interaction tasks on mobile devices. We identify challenges related to (a) intentional and unintentional touches outside of onscreen targets and (b) recognition of drawn gestures, that both indicate a need to design tailored interaction for children to accommodate and overcome these challenges.

[1]  Tracy Anne Hammond,et al.  A Sketch Recognition System for Recognizing Free-Hand Course of Action Diagrams , 2010, IAAI.

[2]  Simeon Keates,et al.  Effect of age and Parkinson's disease on cursor positioning using a mouse , 2005, Assets '05.

[3]  Carly Shuler,et al.  Pockets of Potential : using mobile technologies to promote children's learning , 2009 .

[4]  Karen Littleton,et al.  It is best to point for young children: a comparison of children's pointing and dragging , 1998 .

[5]  Louis Vuurpijl,et al.  Iconic and multi-stroke gesture recognition , 2009, Pattern Recognit..

[6]  Isabelle Guyon,et al.  UNIPEN project of on-line data exchange and recognizer benchmarks , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[7]  Yvonne Rogers,et al.  Children designing together on a multi-touch tabletop: an analysis of spatial orientation and user interactions , 2009, IDC.

[8]  Tom Jones,et al.  An Empirical Study of Children's Use of Computer Pointing Devices , 1991 .

[9]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.

[10]  Janet C. Read,et al.  PENS BEHAVING BADLY – USABILITY OF PENS AND GRAPHICS TABLETS FOR TEXT ENTRY WITH CHILDREN , 2002 .

[11]  Brad A. Myers,et al.  Maximizing the guessability of symbolic input , 2005, CHI Extended Abstracts.

[12]  Namsoo Shin,et al.  Effects of Handheld Games on Students Learning in Mathematics , 2006, ICLS.

[13]  Toward Comparing the Touchscreen Interaction Patterns of Kids and Adults , 2012 .

[14]  Lorna McKnight,et al.  Children's Interaction with Mobile Touch-Screen Devices: Experiences and Guidelines for Design , 2010, Int. J. Mob. Hum. Comput. Interact..

[15]  Lisa Anthony,et al.  $N-protractor: a fast and accurate multistroke recognizer , 2012, Graphics Interface.

[16]  M. Sheelagh T. Carpendale,et al.  Gestures in the wild: studying multi-touch gesture sequences on interactive tabletop exhibits , 2011, CHI.

[17]  Meredith Ringel Morris,et al.  Experiences with and observations of direct-touch tabletops , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[18]  Daniel Vogel,et al.  Estimating the Perceived Difficulty of Pen Gestures , 2011, INTERACT.

[19]  Yvonne Rogers,et al.  Around the table: are multiple-touch surfaces better than single-touch for children's collaborative interactions? , 2009, CSCL.

[20]  Lisa Anthony,et al.  A lightweight multistroke recognizer for user interface prototypes , 2010, Graphics Interface.

[21]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[22]  Yang Li Gesture search: a tool for fast mobile data access , 2010, UIST '10.

[23]  Afke Donker,et al.  Aiming and clicking in young children's use of the computer mouse , 2007, Comput. Hum. Behav..

[24]  A. Richard Newton,et al.  Recognition and beautification of multi-stroke symbols in digital ink , 2005, Comput. Graph..

[25]  Michael Perrone,et al.  Confidence-scoring post-processing for off-line handwritten-character recognition verification , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[26]  Daniel Vogel,et al.  Direct Pen Interaction With a Conventional Graphical User Interface , 2010, Hum. Comput. Interact..

[27]  I. Scott MacKenzie,et al.  Accuracy measures for evaluating computer pointing devices , 2001, CHI.

[28]  Cathleen Norris,et al.  Envisioning the Handheld-Centric Classroom , 2004 .

[29]  Elizabeth Foss,et al.  Clear Panels: a technique to design mobile application interactivity , 2010, Conference on Designing Interactive Systems.

[30]  Allison Druin,et al.  Differences in pointing task performance between preschool children and adults using mice , 2004, TCHI.

[31]  Kori Inkpen Quinn,et al.  Drag-and-drop versus point-and-click mouse interaction styles for children , 2001, TCHI.