The Kinematic Theory Produces Human-Like Stroke Gestures

We show that the Kinematic Theory produces synthesized stroke gestures that “look and feel” the same and hold the same statistical characteristics as human-generated gestures. Previous research in this vein has conducted such comparison from the classification accuracy performance, which is a legitimate though indirect measure. In this article, we synthesized two well-known public datasets comprising unistroke and multistroke gestures. We then compared geometric, kinematic, and articulation aspects of human and synthetic gestures, and found no practical differences between both populations. We also conducted an online survey involving 236 participants and found that it is very difficult to tell human and synthetic gestures apart. We can finally be confident that synthesized gestures are actually reflective of how users produce stroke gestures. In sum, this work enables a deeper understanding of synthetic gestures’ production, which can inform the design of better gesture sets and development of more accurate recognizers. Author

[1]  R Plamondon,et al.  Studying the variability of handwriting patterns using the Kinematic Theory. , 2009, Human movement science.

[2]  Lisa Anthony,et al.  $N-protractor: a fast and accurate multistroke recognizer , 2012, Graphics Interface.

[3]  I. Scott MacKenzie,et al.  Accuracy measures for evaluating computer pointing devices , 2001, CHI.

[4]  R. Plamondon,et al.  A multi-level representation paradigm for handwriting stroke generation. , 2006, Human movement science.

[5]  Radu-Daniel Vatavu,et al.  Understanding the consistency of users' pen and finger stroke gesture articulation , 2013, Graphics Interface.

[6]  Réjean Plamondon,et al.  Training of On-Line Handwriting Text Recognizers with Synthetic Text Generated Using the Kinematic Theory of Rapid Human Movements , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[7]  Timothy D. Lee,et al.  Motor Control and Learning: A Behavioral Emphasis , 1982 .

[8]  Radu-Daniel Vatavu,et al.  Relative accuracy measures for stroke gestures , 2013, ICMI '13.

[9]  Christian O'Reilly,et al.  Neuromuscular Representation and Synthetic Generation of Handwritten Whiteboard Notes , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[10]  Sascha E. Engelbrecht,et al.  Minimum Principles in Motor Control. , 2001, Journal of mathematical psychology.

[11]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.

[12]  Shumin Zhai,et al.  A comparative evaluation of finger and pen stroke gestures , 2012, CHI.

[13]  Réjean Plamondon,et al.  Personal digital bodyguards for e-security, e-learning and e-health: A prospective survey , 2018, Pattern Recognit..

[14]  Réjean Plamondon,et al.  The kinematic theory and minimum principles in motor control : a conceptual comparison , 2008 .

[15]  Atau Tanaka,et al.  Adaptive Gesture Recognition with Variation Estimation for Interactive Systems , 2014, ACM Trans. Interact. Intell. Syst..

[16]  Réjean Plamondon,et al.  A kinematic theory of rapid human movements , 1995, Biological Cybernetics.

[17]  Wolfgang Konen,et al.  Gesture recognition on few training data using Slow Feature Analysis and parametric bootstrap , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[18]  Réjean Plamondon,et al.  Development of a Sigma-Lognormal representation for on-line signatures , 2009, Pattern Recognit..

[19]  Yang Li,et al.  CrowdLearner: rapidly creating mobile recognizers using crowdsourcing , 2013, UIST.

[20]  Charles E. Wright,et al.  Generalized Motor Programs: Reexamining Claims of Effector Independence in Writing , 2018, Attention and Performance XIII.

[21]  Réjean Plamondon,et al.  A kinematic theory of rapid human movements , 1995, Biological Cybernetics.

[22]  Réjean Plamondon,et al.  Modelling velocity profiles of rapid movements: a comparative study , 1993, Biological Cybernetics.

[23]  Thad Starner,et al.  MAGIC summoning: towards automatic suggesting and testing of gestures with low probability of false positives during use , 2013, J. Mach. Learn. Res..

[24]  Yann LeCun,et al.  Reverse TDNN: An Architecture For Trajectory Generation , 1991, NIPS.

[25]  Stephen Grossberg,et al.  The Vite Model: A Neural Command Circuit for Generating Arm and Articulator Trajectories, , 1988 .

[26]  Yang Li,et al.  Gesture script: recognizing gestures and their structure using rendering scripts and interactively trained parts , 2014, CHI.

[27]  P. J. G. Keuss,et al.  Motor aspects of handwriting : approaches to movement in graphic behavior , 1984 .

[28]  Laurent Miclet,et al.  Synthetic On-line Handwriting Generation by Distortions and Analogy , 2007 .

[29]  Julian Fiérrez,et al.  Synthetic on-line signature generation. Part II: Experimental validation , 2012, Pattern Recognit..

[30]  Réjean Plamondon,et al.  An interactive system for the automatic generation of huge handwriting databases from a few specimens , 2008, 2008 19th International Conference on Pattern Recognition.

[31]  Christian O'Reilly,et al.  Recent developments in the study of rapid human movements with the kinematic theory: Applications to handwriting and signature synthesis , 2014, Pattern Recognit. Lett..

[32]  James Arvo,et al.  Equation entry and editing via handwriting and gesture recognition , 2001, Behav. Inf. Technol..

[33]  Richard E. Ladner,et al.  Usable gestures for blind people: understanding preference and performance , 2011, CHI.

[34]  Christian O'Reilly,et al.  The lognormal handwriter: learning, performing, and declining , 2013, Front. Psychol..

[35]  Alejandro Héctor Toselli,et al.  Context-Aware Gestures for Mixed-Initiative Text Editing UIs , 2015, Interact. Comput..

[36]  Stefan Schaal,et al.  Dynamics systems vs. optimal control--a unifying view. , 2007, Progress in brain research.

[37]  R. Plamondon,et al.  The limit profile of a rapid movement velocity. , 2010, Human movement science.

[38]  Shumin Zhai,et al.  Using strokes as command shortcuts: cognitive benefits and toolkit support , 2009, CHI.

[39]  M. Landy,et al.  Statistical decision theory and trade-offs in the control of motor response. , 2003, Spatial vision.

[40]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[41]  Luis A. Leiva,et al.  Gestures à Go Go , 2015, ACM Trans. Intell. Syst. Technol..

[42]  Jun Nakanishi,et al.  Learning Movement Primitives , 2005, ISRR.

[43]  Christian O'Reilly,et al.  Synthetic Handwritten Gesture Generation Using Sigma-Lognormal Model for Evolving Handwriting Classifiers , 2010 .

[44]  J. Hollerbach An oscillation theory of handwriting , 2004, Biological Cybernetics.

[45]  Réjean Plamondon,et al.  Improving sigma-lognormal parameter extraction , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[46]  Shumin Zhai,et al.  Foundational Issues in Touch-Surface Stroke Gesture Design - An Integrative Review , 2012, Found. Trends Hum. Comput. Interact..

[47]  Radu-Daniel Vatavu,et al.  Gestures as point clouds: a $P recognizer for user interface prototypes , 2012, ICMI '12.