A Rapid Prototyping Approach to Synthetic Data Generation for Improved 2D Gesture Recognition

Training gesture recognizers with synthetic data generated from real gestures is a well known and powerful technique that can significantly improve recognition accuracy. In this paper we introduce a novel technique called gesture path stochastic resampling (GPSR) that is computationally efficient, has minimal coding overhead, and yet despite its simplicity is able to achieve higher accuracy than competitive, state-of-the-art approaches. GPSR generates synthetic samples by lengthening and shortening gesture subpaths within a given sample to produce realistic variations of the input via a process of nonuniform resampling. As such, GPSR is an appropriate rapid prototyping technique where ease of use, understandability, and efficiency are key. Further, through an extensive evaluation, we show that accuracy significantly improves when gesture recognizers are trained with GPSR synthetic samples. In some cases, mean recognition errors are reduced by more than 70%, and in most cases, GPSR outperforms two other evaluated state-of-the-art methods.

[1]  Muriel Visani,et al.  Generation of learning samples for historical handwriting recognition using image degradation , 2013, HIP '13.

[2]  Andrew W. Fitzgibbon,et al.  The Joint Manifold Model for Semi-supervised Multi-valued Regression , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[3]  Shumin Zhai,et al.  Using strokes as command shortcuts: cognitive benefits and toolkit support , 2009, CHI.

[4]  Ryszard S. Choraś Image Processing and Communications Challenges 3 - 3rd International Conference, IP&C 2011, Proceedings , 2011, IP&C.

[5]  Ken Perlin,et al.  An image synthesizer , 1988 .

[6]  Joseph J. LaViola,et al.  Streamlined and accurate gesture recognition with Penny Pincher , 2016, Comput. Graph..

[7]  Horst Bunke,et al.  Offline Handwriting Recognition Using Synthetic Training Data Produced By Means Of A Geometrical Distortion Model , 2004, Int. J. Pattern Recognit. Artif. Intell..

[8]  Beryl Plimmer,et al.  The Power of Automatic Feature Selection: Rubine on Steroids , 2010, SBIM.

[9]  Shumin Zhai,et al.  Modeling human performance of pen stroke gestures , 2007, CHI.

[10]  Horst Bunke,et al.  Generation and Use of Synthetic Training Data in Cursive Handwriting Recognition , 2003, IbPRIA.

[11]  Lisa Anthony,et al.  $N-protractor: a fast and accurate multistroke recognizer , 2012, Graphics Interface.

[12]  Masaki Nakagawa,et al.  The Impact of Large Training Sets on the Recognition Rate of Offline Japanese Kanji Character Classifiers , 2002, Document Analysis Systems.

[13]  Daniel Vogel,et al.  Estimating the Perceived Difficulty of Pen Gestures , 2011, INTERACT.

[14]  Venu Govindaraju,et al.  Phrase-based correction model for improving handwriting recognition accuracies , 2009, Pattern Recognit..

[15]  Dit-Yan Yeung,et al.  Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR International Workshops, SSPR 2006 and SPR 2006, Hong Kong, China, August 17-19, 2006, Proceedings , 2006, SSPR/SPR.

[16]  R. Plamondon,et al.  A multi-level representation paradigm for handwriting stroke generation. , 2006, Human movement science.

[17]  Radu-Daniel Vatavu,et al.  Gestures as point clouds: a $P recognizer for user interface prototypes , 2012, ICMI '12.

[18]  Réjean Plamondon,et al.  Improving sigma-lognormal parameter extraction , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[19]  Radu-Daniel Vatavu,et al.  Relative accuracy measures for stroke gestures , 2013, ICMI '13.

[20]  J. Herold,et al.  The 1¢ Recognizer: a fast, accurate, and easy-to-implement handwritten gesture recognition technique , 2012, SBIM '12.

[21]  Michael H. Kutner Applied Linear Statistical Models , 1974 .

[22]  Horst Bunke Template-based Synthetic Handwriting Generation for the Training of Recognition Systems , 2005 .

[23]  Atau Tanaka,et al.  Adaptive Gesture Recognition with Variation Estimation for Interactive Systems , 2014, ACM Trans. Interact. Intell. Syst..

[24]  Stephanie Ludi,et al.  Using Off-Line Features and Synthetic Data for On-Line Handwritten Math Symbol Recognition , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[25]  Réjean Plamondon,et al.  A kinematic theory of rapid human movements , 2004, Biological Cybernetics.

[26]  Christian O'Reilly,et al.  Neuromuscular Representation and Synthetic Generation of Handwritten Whiteboard Notes , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[27]  J. J. Higgins,et al.  The aligned rank transform for nonparametric factorial analyses using only anova procedures , 2011, CHI.

[28]  Yang Li,et al.  Protractor: a fast and accurate gesture recognizer , 2010, CHI.

[29]  Radu-Daniel Vatavu,et al.  The effect of sampling rate on the performance of template-based gesture recognizers , 2011, ICMI '11.

[30]  RubineDean Specifying gestures by example , 1991 .

[31]  Thomas F. Stahovich,et al.  SpeedSeg: A technique for segmenting pen strokes using pen speed , 2011, Comput. Graph..

[32]  Daniel P. Lopresti,et al.  Document analysis systems V : 5th International Workshop, DAS 2002, Princeton, NJ, USA, August 19-21, 2002 : proceedings , 2002 .

[33]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[34]  Zaher Al Aghbari,et al.  Synthizing Handwritten Arabic Text Using Active Shape Models , 2011, IP&C.

[35]  Réjean Plamondon,et al.  A kinematic theory of rapid human movements , 1995, Biological Cybernetics.

[36]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[37]  Hwan-Gue Cho,et al.  A New Synthesizing Method for Handwriting Korean Scripts , 1998, Int. J. Pattern Recognit. Artif. Intell..

[38]  Terry Caelli Structural, syntactic, and statistical pattern recognition : joint IAPR International Workshops SSPR 2002 and SPR 2002, Windsor, Ontario, Canada, August 6-9, 2002 : proceedings , 2002 .

[39]  José A. Rodríguez-Serrano,et al.  Synthesizing queries for handwritten word image retrieval , 2012, Pattern Recognit..

[40]  Lisa Anthony,et al.  A lightweight multistroke recognizer for user interface prototypes , 2010, Graphics Interface.

[41]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[42]  Joseph J. LaViola,et al.  A $-Family Friendly Approach to Prototype Selection , 2016, IUI.

[43]  Julian Fierrez,et al.  Synthetic generation of handwritten signatures based on spectral analysis , 2009, Defense + Commercial Sensing.

[44]  Harold Mouchère,et al.  Towards Handwritten Mathematical Expression Recognition , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[45]  Venu Govindaraju,et al.  Synthetic handwritten CAPTCHAs , 2009, Pattern Recognit..

[46]  Erland Jonsson,et al.  A Synthetic Fraud Data Generation Methodology , 2002, ICICS.

[47]  Horst Bunke,et al.  Generation of synthetic training data for an HMM-based handwriting recognition system , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[48]  Luis A. Leiva,et al.  Gestures à Go Go , 2015, ACM Trans. Intell. Syst. Technol..

[49]  John Bennett,et al.  The effect of large training set sizes on online Japanese Kanji and English cursive recognizers , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[50]  Yang Li,et al.  Gesture script: recognizing gestures and their structure using rendering scripts and interactively trained parts , 2014, CHI.

[51]  Horst Bunke,et al.  Off-Line, Handwritten Numeral Recognition by Perturbation Method , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Ioannis Pratikakis,et al.  A segmentation-free approach for keyword search in historical typewritten documents , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[53]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.

[54]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[55]  Rafael Llobet,et al.  Training Set Expansion in Handwritten Character Recognition , 2002, SSPR/SPR.

[56]  George Labahn,et al.  Tools for the efficient generation of hand-drawn corpora based on context-free grammars , 2009, SBIM '09.