Hybrid HMM/ANN Models for Bimodal Online and Offline Cursive Word Recognition

The recognition performance of current automatic offline handwriting transcription systems is far from being perfect. This is the reason why there is a growing interest in assisted transcription systems, which are more efficient than correcting by hand an automatic transcription. A recent approach to interactive transcription involves multimodal recognition, where the user can supply an online transcription of some of the words. In this paper, a description of the bimodal engine, which entered the "Bi-modal Handwritten Text Recognition" contest organized during the 2010 ICPR, is presented. The proposed recognition system uses Hidden Markov Models hybridized with neural networks (HMM/ANN) for both offline and online input. The N-best word hypothesis scores for both the offline and the online samples are combined using a log-linear combination, achieving very satisfying results.