Unsupervised writer adaptation of whole-word HMMs with application to word-spotting

In this paper we propose a novel approach for writer adaptation in a handwritten word-spotting task. The method exploits the fact that the semi-continuous hidden Markov model separates the word model parameters into (i) a codebook of shapes and (ii) a set of word-specific parameters. Our main contribution is to employ this property to derive writer-specific word models by statistically adapting an initial universal codebook to each document. This process is unsupervised and does not even require the appearance of the keyword(s) in the searched document. Experimental results show an increase in performance when this adaptation technique is applied. To the best of our knowledge, this is the first work dealing with adaptation for word-spotting. The preliminary version of this paper obtained an IBM Best Student Paper Award at the 19th International Conference on Pattern Recognition.

[1]  Alex Waibel,et al.  Readings in speech recognition , 1990 .

[2]  Kengo Terasawa,et al.  Eigenspace method for text retrieval in historical document images , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[3]  Yee Whye Teh,et al.  Making Latin Manuscripts Searchable using gHMMs , 2004, NIPS.

[4]  Dan S. Bloomberg,et al.  Word spotting in scanned images using hidden Markov models , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Anil K. Jain,et al.  Writer Adaptation for Online Handwriting Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Richard M. Schwartz,et al.  A compact model for speaker-adaptive training , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[7]  R. Manmatha,et al.  A Statistical Approach to Retrieving Historical Manuscript Images without Recognition , 2003 .

[8]  Mark J. F. Gales,et al.  The generation and use of regression class trees for MLLR adaptation , 1996 .

[9]  Alan F. Smeaton,et al.  Word matching using single closed contours for indexing handwritten historical documents , 2006, International Journal of Document Analysis and Recognition (IJDAR).

[10]  Robert Sabourin,et al.  An HMM-Based Approach for Off-Line Unconstrained Handwritten Word Modeling and Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Gernot A. Fink,et al.  Unsupervised Estimation of Writing Style Models for Improved Unconstrained Off-line Handwriting Recognition , 2006 .

[12]  Josep Lladós,et al.  Unsupervised writer style adaptation for handwritten word spotting , 2008, 2008 19th International Conference on Pattern Recognition.

[13]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[14]  Philip C. Woodland,et al.  Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models , 1995, Comput. Speech Lang..

[15]  R. Manmatha,et al.  Word spotting for historical documents , 2006, International Journal of Document Analysis and Recognition (IJDAR).

[16]  R. Manmatha,et al.  Word image matching using dynamic time warping , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Florent Perronnin,et al.  A similarity measure between unordered vector sets with application to image categorization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Oscar E. Agazzi,et al.  Keyword Spotting in Poorly Printed Documents using Pseudo 2-D Hidden Markov Models , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Emmanuel Augustin,et al.  Hidden Markov Model Based Word Recognition and Its Application to Legal Amount Reading on French Checks , 1998, Comput. Vis. Image Underst..

[20]  Alvin F. Martin,et al.  The DET curve in assessment of detection task performance , 1997, EUROSPEECH.

[21]  Özgür Ulusoy,et al.  Content-based retrieval of historical Ottoman documents stored as textual images , 2004, IEEE Transactions on Image Processing.

[22]  Kate Knill,et al.  Speaker dependent keyword spotting for accessing stored speech , 1994 .

[23]  David A. Forsyth,et al.  Searching Off-line Arabic Documents , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Jin Hyung Kim,et al.  Print keyword spotting with dynamically synthesized pseudo 2D HMMs , 2004, Pattern Recognit. Lett..

[25]  Xuedong Huang,et al.  Semi-continuous hidden Markov models for speech signals , 1990 .

[26]  Richard Rose,et al.  A hidden Markov model based keyword recognition system , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[27]  Andrew Zisserman,et al.  Efficient Visual Search of Videos Cast as Text Retrieval , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Marcus Liwicki,et al.  A writer identification system for on-line whiteboard data , 2008, Pattern Recognit..

[29]  L. R. Rabiner,et al.  On the use of dynamic time warping for word spotting and connected word recognition , 1981, The Bell System Technical Journal.

[30]  Edward M. Riseman,et al.  Word spotting: a new approach to indexing handwriting , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  José A. Rodríguez-Serrano,et al.  Handwritten word-spotting using hidden Markov models and universal vocabularies , 2009, Pattern Recognit..

[32]  Samy Bengio,et al.  Writer adaptation techniques in HMM based Off-Line Cursive Script Recognition , 2002, Pattern Recognit. Lett..

[33]  Joshua Alspector,et al.  A Line-Oriented Approach to Word Spotting in Handwritten Documents , 2000, Pattern Analysis & Applications.

[34]  Yuzuru Tanaka,et al.  Locality Sensitive Pseudo-Code for Document Images , 2007 .

[35]  Sargur N. Srihari,et al.  Writer adaptation in off-line Arabic handwriting recognition , 2008, Electronic Imaging.

[36]  Kumar Chellapilla,et al.  Personalized handwriting recognition via biased regularization , 2006, ICML.

[37]  Harold Mouchère,et al.  Writer Style Adaptation in Online Handwriting Recognizers by a Fuzzy Mechanism Approach: the Adapt Method , 2007, Int. J. Pattern Recognit. Artif. Intell..

[38]  Roland Kuhn,et al.  Rapid speaker adaptation in eigenvoice space , 2000, IEEE Trans. Speech Audio Process..

[39]  Sargur N. Srihari,et al.  Search engine for handwritten documents , 2005, IS&T/SPIE Electronic Imaging.

[40]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[41]  Lambert Schomaker,et al.  Handwritten-Word Spotting Using Biologically Inspired Features , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.