Reducing language barriers for tourists using handwriting recognition enabled mobile application

Modern mobile devices and networks permit to develop applications that reduce the language barrier for tourists and visitors in foreign countries. This paper describes a mobile application built on a distributed architecture that allows tourists to obtain additional information about location names and menu entries in Arabic language. By simply taking a photo, translation and additional information are displayed to users in their preferred language. While several mobile applications exist to translate the texts into photos, the present application offers the advantage of translating both handwritten and printed texts. The work was focused on two important aspects: the recognition system and the mobile application. The major challenge was to build a recognition system able to recognize handwritten or printed texts of various writing styles or fonts. Therefore, we propose an original approach consisting of recognizing handwritten and printed words using the Balamand-ENST handwriting recognition system, trained on handwritten texts. Experiments conducted on a collected database have shown that the recognition system trained on handwritten text provides excellent performance when used in recognizing printed text. This recognition system and its performance in both modalities are briefly described in this paper. In addition, a full description on how the mobile application was designed and its functionality is presented.

[1]  Richard M. Schwartz,et al.  Multilingual Machine Printed OCR , 2001, Int. J. Pattern Recognit. Artif. Intell..

[2]  Volker Märgner,et al.  Arabic Handwriting Recognition Competition , 2005, ICDAR.

[3]  Chafic Mokbel,et al.  Combining Slanted-Frame Classifiers for Improved HMM-Based Arabic Handwriting Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Chafic Mokbel,et al.  Arabic handwriting recognition using baseline dependant features and hidden Markov modeling , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[5]  Volker Märgner,et al.  HMM based approach for handwritten arabic word recognition using the IFN/ENIT - database , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  Gérard Chollet,et al.  Multimodal user interfaces for a travel assistant , 2003, IHM '03.

[8]  Venu Govindaraju,et al.  Offline Arabic handwriting recognition: a survey , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Chafic Mokbel,et al.  Dynamic and Contextual Information in HMM Modeling for Handwritten Word Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.