Iconic and multi-stroke gesture recognition

Many handwritten gestures, characters, and symbols comprise multiple pendown strokes separated by penup strokes. In this paper, a large number of features known from the literature are explored for the recognition of such multi-stroke gestures. Features are computed from a global gesture shape. From its constituent strokes, the mean and standard deviation of each feature are computed. We show that using these new stroke-based features, significant improvements in classification accuracy can be obtained between 10% and 50% compared to global feature representations. These results are consistent over four different databases, containing iconic pen gestures, handwritten symbols, and upper-case characters. Compared to two other multi-stroke recognition techniques, improvements between 25% and 39% are achieved, averaged over all four databases.

[1]  Joaquim A. Jorge,et al.  Experimental evaluation of an on-line scribble recognizer , 2001, Pattern Recognit. Lett..

[2]  James S. Lipscomb.  A trainable gesture recognizer , 1991, Pattern Recognit..

[3]  Horst Bunke,et al.  A full English sentence database for off-line handwriting recognition , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[4]  Philip R. Cohen,et al.  QuickSet: multimodal interaction for distributed applications , 1997, MULTIMEDIA '97.

[5]  Beat Signer,et al.  iGesture: A General Gesture Recognition Framework , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[6]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[7]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[8]  Ari Visa,et al.  Comparison of Combined Shape Descriptors for Irregular Objects , 1997, BMVC.

[9]  David Goldberg,et al.  Touch-typing with a stylus , 1993, INTERCHI.

[10]  Stefan Knerr,et al.  The IRESTE On/Off (IRONOFF) dual handwriting database , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[11]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[12]  Zhengxing Sun,et al.  An experimental comparison of machine learning for adaptive sketch recognition , 2007, Appl. Math. Comput..

[13]  Joseph J. LaViola,et al.  A Practical Approach for Writer-Dependent Symbol Recognition Using a Writer-Independent Symbol Recognizer , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Louis Vuurpijl,et al.  Finding structure in diversity: a hierarchical clustering method for the categorization of allographs in handwriting , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[15]  Masaki Nakagawa,et al.  Two on-line Japanese character databases in Unipen format , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[16]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  J. IIVARINENHelsinki.  Efficiency of Simple Shape Descriptors , 1997 .

[18]  Isabelle Guyon,et al.  UNIPEN project of on-line data exchange and recognizer benchmarks , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[19]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[20]  James A. Landay,et al.  Interactive sketching for the early stages of user interface design , 1995, CHI '95.

[21]  Masaki Nakagawa,et al.  Collection of on-line handwritten Japanese character pattern databases and their analyses , 2004, Document Analysis and Recognition.

[22]  Selwyn Piramuthu,et al.  Artificial Intelligence and Information Technology Evaluating feature selection methods for learning in data mining applications , 2004 .

[23]  M. Pechwitz,et al.  IFN/ENIT: database of handwritten arabic words , 2002 .

[24]  Ivan Herman,et al.  Advances in Computer Graphics , 1991, EurographicSeminars.

[25]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  Erkki Oja,et al.  Experiments with adaptation strategies for a prototype-based recognition system for isolated handwritten characters , 2001, International Journal on Document Analysis and Recognition.

[28]  James A. Landay,et al.  SATIN: a toolkit for informal ink-based applications , 2000, UIST '00.

[29]  Siska Fitrianie,et al.  A Visual Communication Language for Crisis Management , 2007 .

[30]  Louis Vuurpijl,et al.  Automatic Allograph Matching in Forensic Writer Identification , 2007, Int. J. Pattern Recognit. Artif. Intell..

[31]  Randall Davis,et al.  Sketch recognition in interspersed drawings using time-based graphical models , 2008, Comput. Graph..

[32]  Don J.M. Willems,et al.  Pen Gestures in Online Map and Photograph Annotation Tasks , 2006 .

[33]  Wei Jiang,et al.  HMM-based on-line multi-stroke sketch recognition , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[34]  Sriganesh Madhvanath,et al.  Experiences in Collection of Handwriting Data for Online Handwriting Recognition in Indic Scripts , 2004, LREC.

[35]  Raghuram Thiagarajan,et al.  Advances in Computer Graphics I , 1986, EurographicSeminars.

[36]  Hongan Wang,et al.  Research on User-Centered Design and Recognition Pen Gestures , 2006, Computer Graphics International.

[37]  Louis Vuurpijl,et al.  The NicIcon database of handwritten icons , 2008, ICFHR 2008.

[38]  Isabelle Guyon,et al.  Design of a neural network character recognizer for a touch terminal , 1991, Pattern Recognit..