Cohort of LSTM and lexicon verification for handwriting recognition with gigantic lexicon

Handwriting recognition state of the art methods are based on Long Short Term Memory (LSTM) recurrent neural networks (RNN) coupled with the use of linguistic knowledge. LSTM RNN presents high raw performance and interesting training properties that allow us to break with the standard method at the state of the art. We present a simple and efficient way to extract from a single training a large number of complementary LSTM RNN, called cohort, combined in a cascade architecture with a lexical verification. This process does not require fine tuning, making it easy to use. Our verification allow to deal quickly and efficiently with gigantic lexicon (over 3 million words). We achieve state of the art results for isolated word recognition with very large lexicon and present novel results for an unprecedented gigantic lexicon.

[1]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[2]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[4]  Clément Chatelain,et al.  Spotting handwritten words and REGEX using a two stage BLSTM-HMM architecture , 2015, Electronic Imaging.

[5]  Lior Wolf,et al.  CNN-N-Gram for HandwritingWord Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Volker Märgner,et al.  ICDAR 2009 Online Arabic Handwriting Recognition Competition , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[7]  Clément Chatelain,et al.  Exploring multiple feature combination strategies with a recurrent neural network architecture for off-line handwriting recognition , 2015, Electronic Imaging.

[8]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[9]  Gerhard Rigoll,et al.  Handwritten address recognition with open vocabulary using character n-grams , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Thorsten Brants,et al.  One billion word benchmark for measuring progress in statistical language modeling , 2013, INTERSPEECH.

[12]  Haikal El Abed,et al.  ICDAR 2009 Handwriting Recognition Competition , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[13]  Venu Govindaraju,et al.  Holistic lexicon reduction for handwritten word recognition , 1996, Electronic Imaging.

[14]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[15]  T. Thireou,et al.  Bidirectional Long Short-Term Memory Networks for Predicting the Subcellular Localization of Eukaryotic Proteins , 2007, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[16]  Hermann Ney,et al.  Open Vocabulary Arabic Handwriting Recognition Using Morphological Decomposition , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[17]  Gernot A. Fink,et al.  Markov models for offline handwriting recognition: a survey , 2009, International Journal on Document Analysis and Recognition (IJDAR).

[18]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[19]  Sriganesh Madhvanath,et al.  HMM-Based Lexicon-Driven and Lexicon-Free Word Recognition for Online Handwritten Indic Scripts , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Horst Bunke,et al.  The IAM-database: an English sentence database for offline handwriting recognition , 2002, International Journal on Document Analysis and Recognition.

[21]  Hermann Ney,et al.  The RWTH Large Vocabulary Arabic Handwriting Recognition System , 2014, 2014 11th IAPR International Workshop on Document Analysis Systems.

[22]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[23]  Anthony J. Robinson,et al.  Forward-backward retraining of recurrent neural networks , 1995, NIPS.

[24]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[25]  Christopher Kermorvant,et al.  Dropout Improves Recurrent Neural Networks for Handwriting Recognition , 2013, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[26]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Sargur N. Srihari,et al.  On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Ching Y. Suen,et al.  A novel cascade ensemble classifier system with a high recognition performance on handwritten digits , 2007, Pattern Recognit..

[29]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Bailing Zhang,et al.  Reliable Classification of Vehicle Types Based on Cascade Classifier Ensembles , 2013, IEEE Transactions on Intelligent Transportation Systems.

[31]  T. Munich,et al.  Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks , 2008, NIPS.

[32]  Clément Chatelain,et al.  A two-stage outlier rejection strategy for numerical field extraction in handwritten documents , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[33]  Robert Sabourin,et al.  Large vocabulary off-line handwriting recognition: A survey , 2003, Pattern Analysis & Applications.

[34]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[35]  Fumitaka Kimura,et al.  Handwritten word recognition using lexicon free and lexicon directed word recognition algorithms , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[36]  Yann LeCun,et al.  The Loss Surfaces of Multilayer Networks , 2014, AISTATS.

[37]  Clément Chatelain,et al.  Segmentation-Driven Recognition Applied to Numerical Field Extraction from Handwritten Incoming Mail Documents , 2006, Document Analysis Systems.

[38]  Venu Govindaraju,et al.  Syntactic methodology of pruning large lexicons in cursive script recognition , 2001, Pattern Recognit..

[39]  Jürgen Schmidhuber,et al.  Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks , 2006, ICML.

[40]  Jürgen Schmidhuber,et al.  Learning Precise Timing with LSTM Recurrent Networks , 2003, J. Mach. Learn. Res..

[41]  Robert Sabourin,et al.  An HMM-Based Approach for Off-Line Unconstrained Handwritten Word Modeling and Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Christopher Kermorvant,et al.  The A2iA French handwriting recognition system at the Rimes-ICDAR2011 competition , 2012, Electronic Imaging.