Template Based Recognition of On-Line Handwriting

Software for recognition of handwriting has been available for several decades now and research on the subject have produced several different strategies for producing competitive recognition accuracies, especially in the case of isolated single characters. The problem of recognizing samples of handwriting with arbitrary connections between constituent characters (emph{unconstrained handwriting}) adds considerable complexity in form of the segmentation problem. In other words a recognition system, not constrained to the isolated single character case, needs to be able to recognize where in the sample one letter ends and another begins. In the research community and probably also in commercial systems the most common technique for recognizing unconstrained handwriting compromise Neural Networks for partial character matching along with Hidden Markov Modeling for combining partial results to string hypothesis. Neural Networks are often favored by the research community since the recognition functions are more or less automatically inferred from a training set of handwritten samples. From a commercial perspective a downside to this property is the lack of control, since there is no explicit information on the types of samples that can be correctly recognized by the system. In a template based system, each style of writing a particular character is explicitly modeled, and thus provides some intuition regarding the types of errors (confusions) that the system is prone to make. Most template based recognition methods today only work for the isolated single character recognition problem and extensions to unconstrained recognition is usually not straightforward. This thesis presents a step-by-step recipe for producing a template based recognition system which extends naturally to unconstrained handwriting recognition through simple graph techniques. A system based on this construction has been implemented and tested for the difficult case of unconstrained online Arabic handwriting recognition with good results. (Less)

[1]  Marc Parizeau,et al.  Segmentation and reconstruction of on-line handwritten scripts , 1998, Pattern Recognit..

[2]  Edward Fredkin,et al.  Trie memory , 1960, Commun. ACM.

[3]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[4]  Erkki Oja,et al.  On-line adaptation in recognition of handwritten alphanumeric characters , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[5]  Eugene H. Ratzlaff.  Methods, reports and survey for the comparison of diverse isolated character recognition results on the UNIPEN database , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[6]  A. Kosmala,et al.  A SYSTEMATIC COMPARISON OF ADVANCED MODELING TECHNIQUES FOR VERY LARGE VOCABULARY ON-LINE CURSIVE HANDWRITING RECOGNITION , 1999 .

[7]  Claus Bahlmann,et al.  Advanced sequence classification techniques applied to online handwriting recognition , 2005 .

[8]  Anders Ericsson,et al.  Core points - a framework for structural parameterization , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[9]  Malcolm I. Heywood,et al.  Towards the on-line recognition of Arabic characters , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[10]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[11]  Réjean Plamondon,et al.  A handwriting model for syntactic recognition of cursive script , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[12]  G.,et al.  SIGNAL REPRESENTATIONS FOR HIDDEN MARKOV MODEL BASED ON-LINE HANDWRITING RECOGNITION , .

[13]  Claudio De Stefano,et al.  A saliency-based multiscale method for on-line cursive handwriting shape description , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[14]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[15]  Éric Anquetil,et al.  Lexical post-processing optimization for handwritten word recognition , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[16]  Giovanni Seni,et al.  DIACRITICAL PROCESSING USING EFFICIENT ACCOUNTING PROCEDURES IN A FORWARD SEARCH , 1999 .

[17]  Cyrus Bakhtiari-Haftlang.  Arabic Online Handwriting Recognition , 2007 .

[18]  Louis Vuurpijl,et al.  Finding structure in diversity: a hierarchical clustering method for the categorization of allographs in handwriting , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[19]  Daming Shi,et al.  A radical approach to handwritten Chinese character recognition using active handwriting models , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Ralph Niels,et al.  Dynamic Time Warping: An Intuitive Way of Handwriting Recognition? , 2004 .

[21]  Yoshua Bengio,et al.  LeRec: A NN/HMM Hybrid for On-Line Handwriting Recognition , 1995, Neural Computation.

[22]  Kalle Åström,et al.  Extension of Affine Shape , 2004, Journal of Mathematical Imaging and Vision.

[23]  Jakob Sternby.  Frame Deformation Energy Matching of On-Line Handwritten Characters , 2005, CIARP.

[24]  Bernadette Dorizzi,et al.  On-line cursive script recognition: A user-adaptive system for word identification , 1996, Pattern Recognit..

[25]  Simon M. Lucas,et al.  Efficient graph-based dictionary search and its application to text-image searching , 2001, Pattern Recognit. Lett..

[26]  Pietro G. Morasso,et al.  Recognition experiments of cursive dynamic handwriting with self-organizing networks , 1993, Pattern Recognit..

[27]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Éric Anquetil,et al.  Perceptual model of handwriting drawing. Application to the handwriting segmentation problem , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[29]  Yannis A. Dimitriadis,et al.  A Hybrid Two-Stage Fuzzy ARTMAP and LVQ Neuro-Fuzzy System for On-Line Handwriting Recognition , 2002, ICANN.

[30]  Louis Vuurpijl,et al.  Verifying the UNIPEN devset , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[31]  Robert H. Kassel,et al.  A comparison of approaches to on-line handwritten character recognition , 1995 .

[32]  S. Ganapathy,et al.  Preprocessing techniques for cursive script word recognition , 1983, Pattern Recognit..

[33]  Marcus Liwicki,et al.  IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[34]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[35]  Yoshua Bengio,et al.  AdaBoosting Neural Networks: Application to on-line Character Recognition , 1997, ICANN.

[36]  Christian Viard-Gaudin,et al.  Using Segmentation Constraints in an Implicit Segmentation Scheme for On-line Word Recognition , 2006 .

[37]  Jakob Sternby,et al.  The recognition graph - language independent adaptable on-line cursive script recognition , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[38]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[39]  Marc Parizeau,et al.  Machine vs humans in a cursive script reading experiment without linguistic knowledge , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[40]  P. Neskovic,et al.  NEURAL NETWORK­BASED CONTEXT DRIVEN RECOGNITION OF ON­LINE CURSIVE SCRIP , 2004 .

[41]  Character recognition experiments using Unipen data , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[42]  Vuokko Vuori,et al.  Clustering writing styles with a self-organizing map , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[43]  Colin Higgins,et al.  Online recognition of connected handwriting by segmentation and template matching , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[44]  Jakob Sternby.  Core Points for Segmentation and Recognition of On-line Cursive Script , 2005 .

[45]  Harry Shum,et al.  Combining shape and physical modelsfor online cursive handwriting synthesis , 2004, International Journal of Document Analysis and Recognition (IJDAR).

[46]  Mohamed A. Ismail,et al.  A graph-based segmentation and feature extraction framework for Arabic text recognition , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[47]  Jin Hyung Kim,et al.  Network-based approach to online cursive script recognition , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[48]  Réjean Plamondon,et al.  Normalizing and restoring on-line handwriting , 1993, Pattern Recognit..

[49]  Éric Anquetil,et al.  Lexicon organization and string edit distance learning for lexical post-processing in handwriting recognition , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[50]  S. Marshall,et al.  Review of shape coding techniques , 1989, Image Vis. Comput..

[51]  Seiichi Uchida,et al.  Online character recognition based on elastic matching and quadratic discrimination , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[52]  Amar Mitiche,et al.  A new representation of character shape and its use in on-line character recognition by a self organizing map , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[53]  Christian Viard-Gaudin,et al.  MS-TDNN with global discriminant trainings , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[54]  Gerhard Rigoll,et al.  Tree-based state clustering using self-organizing principles for large vocabulary on-line handwriting recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[55]  Gaofeng Qian.  An engine for cursive handwriting interpretation , 1999, Proceedings 11th International Conference on Tools with Artificial Intelligence.

[56]  Ching Y. Suen,et al.  The State of the Art in Online Handwriting Recognition , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Leon N. Cooper,et al.  Interactive Parts Model: An Application to Recognition of On-line Cursive Script , 2000, NIPS.

[58]  Marcus Liwicki,et al.  HMM-Based On-Line Recognition of Handwritten Whiteboard Notes , 2006 .

[59]  Jakob Sternby.  On-Line Signature Verification by Explicit Solution to the Point Correspondence Problem , 2004 .

[60]  Michael T. Goodrich.  Efficient piecewise-linear function approximation using the uniform metric: (preliminary version) , 1994, SCG '94.

[61]  Eric Lecolinet,et al.  A Survey of Methods and Strategies in Character Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Kasturi R. Varadarajan.  Approximating monotone polygonal curves using the uniform metric , 1996, SCG '96.

[63]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[64]  John T. Favata.  Offline General Handwritten Word Recognition Using an Approximate BEAM Matching Algorithm , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[65]  Masaki Nakagawa,et al.  'Online recognition of Chinese characters: the state-of-the-art , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Marcus Liwicki,et al.  Feature selection for on-line handwriting recognition of whiteboard notes , 2007 .

[67]  Jin Hyung Kim,et al.  Ligature Modeling for Online Cursive Script Recognition , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[68]  John Bennett,et al.  The effect of large training set sizes on online Japanese Kanji and English cursive recognizers , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[69]  Jakob Sternby.  Prototype Selection Methods for On-line HWR , 2006 .

[70]  Van Nostrand,et al.  Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , 1967 .

[71]  Isabelle Guyon,et al.  On-line cursive script recognition using time-delay neural networks and hidden Markov models , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[72]  Giovanni Seni,et al.  Large Vocabulary Recognition of On-Line Handwritten Cursive Words , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Sargur N. Srihari,et al.  On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Lionel Prevost,et al.  Modelizing character allographs in omni-scriptor frame: a new non-supervised clustering algorithm , 2000, Pattern Recognit. Lett..

[75]  Sriganesh Madhvanath,et al.  Principal component analysis for online handwritten character recognition , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[76]  Nizar Habash,et al.  Online Arabic Handwriting Recognition Using Hidden Markov Models , 2006 .

[77]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[78]  Timothy F. Cootes,et al.  Active Shape Models and the shape approximation problem , 1996, Image Vis. Comput..

[79]  Alex Waibel,et al.  A Fast Search Technique for Large Vocabulary On-Line Handwriting Recognition , 1998 .

[80]  S. Lucas.  Efficient best-first dictionary search given graph-based input , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[81]  F. Preparata,et al.  Segmenting Handwritten Signatures at Their Perceptually Important Points , 1993 .

[82]  Jianying Hu,et al.  Writer independent on-line handwriting recognition using an HMM approach , 2000, Pattern Recognit..

[83]  Anders Eriksson,et al.  On the Bijectivity of Thin-Plate Splines , 2005 .

[84]  Jakob Sternby.  Core Points - Variable and Reduced Parameterization for Symbol Recognition , 2005 .

[85]  Mou-Yen Cheii,et al.  Variable Duration Hidden Markov Model and Morphological Segmentation for Handwritten Word Recognition , 1993 .

[86]  Éric Anquetil,et al.  Use of a Confusion Network to Detect and Correct Errors in an On-Line Handwritten Sentence Recognition System , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[87]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[88]  Masaki Nakagawa,et al.  Robust and highly customizable recognition of online handwritten Japanese characters , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[89]  Anil K. Jain,et al.  Online handwriting recognition using multiple pattern class models , 2000 .

[90]  Kazumi Odaka,et al.  On-line cursive Kanji character recognition using stroke-based affine transformation , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[91]  Stan Sclaroff,et al.  Online and offline character recognition using alignment to prototypes , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[92]  Marcus Liwicki,et al.  Combining On-Line and Off-Line Systems for Handwriting Recognition , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[93]  Masaki Nakagawa,et al.  On-line handwritten character pattern database sampled in a sequence of sentences without any writing instructions , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[94]  Venu Govindaraju,et al.  Equivalence of Different Methods for Slant and Skew Corrections in Word Recognition Applications , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[95]  Claus Bahlmann,et al.  Online handwriting recognition with support vector machines - a kernel approach , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[96]  Claus Bahlmann,et al.  The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[97]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[98]  Dinesh Mandalapu,et al.  Active-DTW : A Generative Classifier that combines Elastic Matching with Active Shape Modeling for Online Handwritten Character Recognition , 2006 .

[99]  Éric Anquetil,et al.  Integration of an on-line handwriting recognition system in a smart phone device , 2002, Object recognition supported by user interaction for service robots.

[100]  Jung-Hsien Chiang,et al.  Neural and Fuzzy Methods in Handwriting Recognition , 1997, Computer.

[101]  Reinhold Häb-Umbach,et al.  Signal representations for hidden Markov model based online handwriting recognition , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[102]  Jorma Laaksonen,et al.  LVQ_PAK: The Learning Vector Quantization Program Package , 1996 .

[103]  Alex Waibel,et al.  NPEN++ : AN ON-LINE HANDWRITING RECOGNITION SYSTEM , 2004 .

[104]  Anil K. Jain,et al.  Template-based online character recognition , 2001, Pattern Recognit..

[105]  Charles C. Tappert,et al.  Cursive Script Recognition by Elastic Matching , 1982, IBM J. Res. Dev..

[106]  Louis Vuurpijl,et al.  TWO-STAGE CHARACTER CLASSIFICATION: A COMBINED APPROACH OF CLUSTERING AND SUPPORT VECTOR CLASSIFIERS , 2000 .

[107]  Jakob Sternby,et al.  Class Dependent Cluster Refinement , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[108]  Thierry Artières,et al.  Conditional Random Fields for Online Handwriting Recognition , 2006 .

[109]  B. Yegnanarayana,et al.  Artificial neural networks for pattern recognition , 1994 .

[110]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[111]  A. Ciobanu,et al.  Handwritten script recognition system for languages with diacritic signs , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[112]  Oh Jong.  An On-Line Handwriting Recognizer with Fisher Matching, Hypotheses Propagation Network and Context C , 2001 .

[113]  David Nahamoo,et al.  A Fast Statistical Mixture Algorithm for On-Line Handwriting Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[114]  Takashi Matsumoto,et al.  The reduction of memory and the improvement of recognition rate for HMM on-line handwriting recognition , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[115]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[116]  Isabelle Guyon,et al.  UNIPEN project of on-line data exchange and recognizer benchmarks , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[117]  Marc Parizeau,et al.  A Fuzzy-Syntactic Approach to Allograph Modeling for Cursive Script Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[118]  Jakob Sternby.  An Additive Single Character Recognition Method , 2006 .

[119]  Stefan Knerr,et al.  The IRESTE On/Off (IRONOFF) dual handwriting database , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[120]  JinHyung Kim,et al.  Data-driven Design of HMM Topology for On-line Handwriting Recognition , 2000 .

[121]  Masaki Nakagawa,et al.  The state of the art in Japanese online handwriting recognition compared to techniques in western handwriting recognition , 2003, Document Analysis and Recognition.

[122]  Abdul Rahim Ahmad,et al.  Online handwriting recognition using support vector machine , 2004, 2004 IEEE Region 10 Conference TENCON 2004..

[123]  Horst Bunke,et al.  Recognition of cursive Roman handwriting: past, present and future , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[124]  Jonas Andersson,et al.  On-line Arabic handwriting recognition with templates , 2009, Pattern Recognit..

[125]  Yoshua Bengio,et al.  Word normalization for on-line handwritten word recognition , 1994 .

[126]  Ethem Alpaydin,et al.  Combining multiple representations and classifiers for pen-based handwritten digit recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[127]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[128]  Jianchang Mao,et al.  An efficient algorithm for matching a lexicon with a segmentation graph , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[129]  M. P. Perrone,et al.  K­MEANS CLUSTERING FOR HIDDEN MARKOV MODEL , 2004 .

[130]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[131]  Giovanni Soda,et al.  Artificial neural networks for document analysis and recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[132]  Jakob Sternby.  Structurally Based Template Matching of On-line Handwritten Characters , 2005, BMVC.