Hidden Markov Models for Off-Line Cursive Handwriting Recognition

Abstract Hidden Markov models nowadays belong to the most widely used statistical models for the challenging task of handwriting recognition in document images. In this chapter, we describe the predominant application given by segmentation-free recognition of handwritten text lines in presence of large vocabularies and multiple writers. A review of the state of the art is provided for feature sequence extraction, character appearance modeling, and language modeling. In particular, we comment on the typical parametrization of hidden Markov models for appearance modeling and discuss efficient dynamic programming solutions for training and recognition. Besides the review of appearance and language modeling, this chapter also provides an introduction to confidence modeling, which is an important topic of current research. Confidence models estimate the reliability of a recognition result that can be used for diverse applications including rejection of unreliable results, writer identification and verification, combination of multiple classifiers, and keyword spotting. Finally, some trends and challenges for future research are identified.

[1]  Marc-Peter Schambach Fast script word recognition with very large vocabulary , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[2]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[3]  Hermann Ney,et al.  Confidence measures for large vocabulary continuous speech recognition , 2001, IEEE Trans. Speech Audio Process..

[4]  Robert Sabourin,et al.  An HMM-Based Approach for Off-Line Unconstrained Handwritten Word Modeling and Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[6]  Angelo Chianese,et al.  Recovering dynamic information from static handwriting , 1993, Pattern Recognit..

[7]  Slava M. Katz,et al.  Estimation of probabilities from sparse data for the language model component of a speech recognizer , 1987, IEEE Trans. Acoust. Speech Signal Process..

[8]  Hermann Ney,et al.  Improved backing-off for M-gram language modeling , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[9]  Volkmar Frinken,et al.  HMM-Based Alignment of Inaccurate Transcriptions for Historical Documents , 2011, 2011 International Conference on Document Analysis and Recognition.

[10]  Frederick Jelinek,et al.  25 Continuous speech recognition: Statistical methods , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[11]  Yee Whye Teh,et al.  Making Latin Manuscripts Searchable using gHMMs , 2004, NIPS.

[12]  Horst Bunke,et al.  Using a Statistical Language Model to Improve the Performance of an HMM-Based Cursive Handwriting Recognition System , 2001, Int. J. Pattern Recognit. Artif. Intell..

[13]  Salvador España Boquera,et al.  Improving Offline Handwritten Text Recognition with Hybrid HMM/ANN Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  George Saon Cursive word recognition using a random field based hidden Markov model , 1999, International Journal on Document Analysis and Recognition.

[15]  Gerhard Rigoll,et al.  Confidence measures for an address reading system , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[16]  Andreas Keller,et al.  Lexicon-free handwritten word spotting using character HMMs , 2012, Pattern Recognit. Lett..

[17]  Torsten Caesar,et al.  Preprocessing and feature extraction for a handwriting recognition system , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[18]  Horst Bunke,et al.  Kernel PCA for HMM-Based Cursive Handwriting Recognition , 2009, CAIP.

[19]  Shijian Lu,et al.  Binarization of historical document images using the local maximum and minimum , 2010, DAS '10.

[20]  Robert Sabourin,et al.  Large vocabulary off-line handwriting recognition: A survey , 2003, Pattern Analysis & Applications.

[21]  Haikal El Abed,et al.  ICDAR 2009 Handwriting Recognition Competition , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[22]  Gernot A. Fink,et al.  Towards Feature Learning for HMM-based Offline Handwriting Recognition , 2011 .

[23]  Biing-Hwang Juang,et al.  A training procedure for verifying string hypotheses in continuous speech recognition , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[24]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[25]  Alejandro Héctor Toselli,et al.  Multimodal interactive transcription of text images , 2010, Pattern Recognit..

[26]  Horst Bunke,et al.  The IAM-database: an English sentence database for offline handwriting recognition , 2002, International Journal on Document Analysis and Recognition.

[27]  Guy Lorette Handwriting recognition or reading? What is the situation at the dawn of the 3rd millenium? , 1999, International Journal on Document Analysis and Recognition.

[28]  Basilios Gatos,et al.  ICFHR 2010 Handwriting Segmentation Contest , 2010, 2010 12th International Conference on Frontiers in Handwriting Recognition.

[29]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[30]  Alfons Juan-Císcar,et al.  Confidence Measures for Error Correction in Interactive Transcription Handwritten Text , 2009, ICIAP.

[31]  Rohit Prasad,et al.  Improvements in BBN's HMM-Based Offline Arabic Handwriting Recognition System , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[32]  Horst Bunke,et al.  Hidden Markov model length optimization for handwriting recognition systems , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[33]  Horst Bunke,et al.  TV-gram language models for offline handwritten text recognition , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[34]  Sargur N. Srihari,et al.  Semi-supervised Learning for Handwriting Recognition , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[35]  Gernot A. Fink,et al.  Markov models for offline handwriting recognition: a survey , 2009, International Journal on Document Analysis and Recognition (IJDAR).

[36]  Francesco Tortorella An Optimal Reject Rule for Binary Classifiers , 2000, SSPR/SPR.

[37]  Simon Thomas,et al.  An Information Extraction Model for Unconstrained Handwritten Documents , 2010, 2010 20th International Conference on Pattern Recognition.

[38]  Marcus Liwicki,et al.  Combining Alignment Results for Historical Handwritten Document Analysis , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[39]  Laurence Likforman-Sulem,et al.  Text line segmentation of historical documents: a survey , 2007, International Journal of Document Analysis and Recognition (IJDAR).

[40]  Jean-Cédric Chappelier,et al.  Offline grammar-based recognition of handwritten sentences , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Juergen Luettin,et al.  OFF-LINE CURSIVE SCRIPT RECOGNITION BASED ON CONTINUOUS DENSITY HMM , 2004 .

[42]  Horst Bunke,et al.  Offline Handwriting Recognition Using Synthetic Training Data Produced By Means Of A Geometrical Distortion Model , 2004, Int. J. Pattern Recognit. Artif. Intell..

[43]  Mounim A. El-Yacoubi,et al.  A Statistical Approach for Phrase Location and Recognition within a Text Line: An Application to Street Name Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Gerhard Rigoll,et al.  Handwritten Address Recognition Using Hidden Markov Models , 2004, Reading and Learning.

[45]  Kenneth M. Sayre,et al.  Machine recognition of handwritten words: A project report , 1973, Pattern Recognit..

[46]  David G. Stork,et al.  Pattern Classification , 1973 .

[47]  José A. Rodríguez-Serrano,et al.  Handwritten word-spotting using hidden Markov models and universal vocabularies , 2009, Pattern Recognit..

[48]  Øivind Due Trier,et al.  Evaluation of Binarization Methods for Document Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Gernot A. Fink,et al.  Toward automatic video-based whiteboard reading , 2004, International Journal of Document Analysis and Recognition (IJDAR).

[50]  Ching Y. Suen,et al.  Thinning Methodologies - A Comprehensive Survey , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Juergen Luettin,et al.  A new normalization technique for cursive handwritten words , 2001, Pattern Recognit. Lett..

[52]  Biing-Hwang Juang,et al.  Maximum likelihood estimation for multivariate mixture observations of markov chains , 1986, IEEE Trans. Inf. Theory.

[53]  Ioannis Pratikakis,et al.  H-DIBCO 2010 - Handwritten Document Image Binarization Competition , 2010, 2010 12th International Conference on Frontiers in Handwriting Recognition.

[54]  Emmanuel Augustin,et al.  Industrial bank check processing: the A2iA CheckReaderTM , 2001, International Journal on Document Analysis and Recognition.

[55]  Sargur N. Srihari,et al.  A system to read names and addresses on tax forms , 1996 .

[56]  Horst Bunke,et al.  Off-line handwriting identification using HMM based recognizers , 2004, ICPR 2004.

[57]  Samy Bengio,et al.  Offline recognition of unconstrained handwritten texts using HMMs and statistical language models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Alessandro L. Koerich Rejection strategies for handwritten word recognition , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[59]  Horst Bunke,et al.  Off-Line Writer Verification: A Comparison of a Hidden Markov Model (HMM) and a Gaussian Mixture Model (GMM) Based System , 2006 .

[60]  Apostolos Antonacopoulos,et al.  Special issue on the analysis of historical documents , 2007, International Journal of Document Analysis and Recognition (IJDAR).

[61]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[62]  Sargur N. Srihari,et al.  On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  Johansson. Stig,et al.  Manual of information to accompany the Lancaster-Oslo : Bergen Corpus of British English, for use with digital computers , 1978 .

[64]  Sargur N. Srihari,et al.  Off-Line Cursive Script Word Recognition , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[65]  Chafic Mokbel,et al.  Arabic handwriting recognition using baseline dependant features and hidden Markov modeling , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[66]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[67]  Claudio M. Privitera,et al.  The segmentation of cursive handwriting: an approach based on off-line recovery of the motor-temporal information , 1999, IEEE Trans. Image Process..

[68]  Michael Perrone,et al.  Confidence modeling for handwriting recognition: algorithms and applications , 2005, International Journal of Document Analysis and Recognition (IJDAR).

[69]  Kaspar Riesen,et al.  Graph Similarity Features for HMM-Based Handwriting Recognition in Historical Documents , 2010, 2010 12th International Conference on Frontiers in Handwriting Recognition.

[70]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[71]  Michael J. Fischer,et al.  The String-to-String Correction Problem , 1974, JACM.

[72]  Horst Bunke,et al.  Recovery of temporal information of cursively handwritten words for on-line recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[73]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[74]  Sargur N. Srihari Handwritten Address Interpretation: A Task of Many Pattern Recognition Problems , 2000, Int. J. Pattern Recognit. Artif. Intell..

[75]  Bernadette Dorizzi,et al.  Rejection measures for handwriting sentence recognition , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[76]  Horst Bunke,et al.  Hidden Markov model-based ensemble methods for offline handwritten text line recognition , 2008, Pattern Recognit..