Snap n' shop: Visual search-based mobile shopping made a breeze by machine and crowd intelligence

The increasing popularity of smartphones has significantly changed the way we live. Today's powerful mobile systems provide us with all kinds of convenient services. Thanks to the wide variety of available apps, it has never been so easy for people to shop, to navigate, and to communicate with others. However, for some tasks we can further improve the user experience by employing newly developed algorithms. In this work, we try to improve visual search based mobile shopping experience by using machine and crowd intelligence. In particular, our system enables precise object selection, which would lead to more accurate visual search results. We also use crowdsourcing to further extend the system's prowess. We conduct experiments on user interface design and retrieval performance, which validate the effectiveness and ease of use of the proposed system. Meanwhile, components in the system are quite modular, allowing the flexibility of adding or improving different modules of the whole system.

[1]  Michael Isard,et al.  Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Bernd Girod,et al.  Mobile Visual Search: Architectures, Technologies, and the Emerging MPEG Standard , 2011, IEEE MultiMedia.

[3]  Huizhong Chen,et al.  The stanford mobile visual search data set , 2011, MMSys.

[4]  Bernd Girod,et al.  Mobile product recognition , 2010, ACM Multimedia.

[5]  Rongrong Ji,et al.  Active query sensing: Suggesting the best query view for mobile visual search , 2012, TOMCCAP.

[6]  Ling-Yu Duan,et al.  Compact descriptors for mobile visual search and MPEG CDVS standardization , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[7]  Eckehard G. Steinbach,et al.  Rapid image retrieval for mobile location recognition , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Yiannis S. Boutalis,et al.  CEDD: Color and Edge Directivity Descriptor: A Compact Descriptor for Image Indexing and Retrieval , 2008, ICVS.

[9]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[10]  Ying Wu,et al.  Mobile Product Image Search by Automatic Query Object Extraction , 2012, ECCV.

[11]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[14]  FuaPascal,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012 .

[15]  K. K. More,et al.  Interactive Multimodal Visual Search on Mobile Device , 2015 .

[16]  Winston H. Hsu,et al.  Large-scale simultaneous multi-object recognition and localization via bottom up search-based approach , 2012, ACM Multimedia.

[17]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.