Image-based mobile service: automatic text extraction and translation

We present a new mobile service for the translation of text from images taken by consumer-grade cell-phone cameras. Such capability represents a new paradigm for users where a simple image provides the basis for a service. The ubiquity and ease of use of cell-phone cameras enables acquisition and transmission of images anywhere and at any time a user wishes, delivering rapid and accurate translation over the phone's MMS and SMS facilities. Target text is extracted completely automatically, requiring no bounding box delineation or related user intervention. The service uses localization, binarization, text deskewing, and optical character recognition (OCR) in its analysis. Once the text is translated, an SMS message is sent to the user with the result. Further novelties include that no software installation is required on the handset, any service provider or camera phone can be used, and the entire service is implemented on the server side.

[1]  Yasuhiko Watanabe,et al.  Translation camera on mobile phone , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[2]  Steven J. Simske,et al.  Security Printing Deterrents : A Comparison of Thermal Ink Jet, Dry Electrophotographic, and Liquid Electrophotographic Printing , 2008 .

[3]  Ying Zhang,et al.  Towards Automatic Sign Translation , 2001, HLT.

[4]  Konrad Tollmar,et al.  Searching the Web with mobile images for location recognition , 2004, CVPR 2004.

[5]  Majid Mirmehdi,et al.  Rectifying perspective views of text in 3D scenes using vanishing points , 2003, Pattern Recognit..

[6]  Alex Waibel,et al.  An automatic sign recognition and translation system , 2001, PUI '01.

[7]  Yasuhiko Watanabe,et al.  Translation camera , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[8]  Horst Bunke,et al.  Handbook of Character Recognition and Document Image Analysis , 1997 .

[9]  Maurizio Pilu,et al.  A light-weight text image processing method for handheld embedded cameras , 2002, BMVC.

[10]  Gérard G. Medioni,et al.  Text segmentation in color images using tensor voting , 2007, Image Vis. Comput..

[11]  Ismail Haritaoglu.  Scene text extraction and translation for handheld devices , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Konrad Tollmar,et al.  IDeixis: image-based Deixis for finding location-based information , 2004, CHI EA '04.

[14]  Abigail Sellen,et al.  The ubiquitous camera: an in-depth study of camera phone use , 2005, IEEE Pervasive Computing.

[15]  Xilin Chen,et al.  Automatic detection and recognition of signs from natural scenes , 2004, IEEE Transactions on Image Processing.

[16]  Sabine Süsstrunk,et al.  Mobile cosmetics advisor: an imaging based mobile service , 2010, Electronic Imaging.

[17]  Sabine Süsstrunk,et al.  Color match: an imaging based mobile cosmetics advisory service , 2008, Mobile HCI.