Mobile-based advertisement information retrieval from images and websites

In the real world, there are a huge amount of advertisement (ad) boards to make customers have a visual awareness of the products or services easily. However, information appearing in the ad boards is so limited that customers always want to know more ad details in a convenient way. In this paper, we present an mobile-based prototype system to automatically extract web ad information from images and websites. After capturing ad images by smartphones and sending them to a remote server, ad image text is recognized by OCR engine, from where ad phrases and keywords are extracted and combined together as queries. Ad web page candidates are then obtained by specific search engines and clustered to remove noises. OCR results are further used to estimate valid ad topic web pages which are pushed back to end users for searching more detailed ad information. Based on the experiments on a real-world ad image dataset collected by ourselves, true ad topic web pages can be found from top-one and top-ten returned pages in about 51.85% and 83.33% query images respectively, which illustrates the effectiveness of the proposed system.

[1]  Cheng-Hsin Hsu,et al.  Building book inventories using smartphones , 2010, ACM Multimedia.

[2]  Jian Fan,et al.  Mobile document scanning and copying , 2010, ACM Multimedia.

[3]  Wen Gao,et al.  Location Discriminative Vocabulary Coding for Mobile Landmark Search , 2012, International Journal of Computer Vision.

[4]  Cheng-Lin Liu,et al.  A Hybrid Approach to Detect and Localize Texts in Natural Scene Images , 2011, IEEE Transactions on Image Processing.

[5]  Alan L. Yuille,et al.  A Time-Efficient Cascade for Real-Time Object Detection: With applications for the visually impaired , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[6]  Anil K. Jain,et al.  Text information extraction in images and video: a survey , 2004, Pattern Recognit..

[7]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[8]  Yang Wang,et al.  JIGSAW: interactive mobile visual search with multimodal queries , 2011, ACM Multimedia.

[9]  David S. Doermann,et al.  Camera-based analysis of text and documents: a survey , 2005, International Journal of Document Analysis and Recognition (IJDAR).

[10]  Xing Xie,et al.  Location sensitive indexing for image-based advertising , 2009, MM '09.

[11]  Cheng-Lin Liu,et al.  Handwritten Japanese Character String Recognition Using Conditional Random Fields , 2009 .