Parallel AP Clustering and Re-ranking for Automatic Image-Text Alignment and Large-Scale Web Image Search

In this paper, an automatic image-text alignment algorithm is developed for achieving more accurate indexing and retrieval of large-scale web images. First, large-scale web pages are crawled, where the informative images and their most relevant auxiliary text blocks are extracted. Second, parallel image clustering is performed to partition large-scale informative web images into a large number of clusters. By grouping the visually-similar (near-duplicate) web images into the same cluster, our parallel image clustering algorithm can significantly reduce the huge uncertainty on the relatedness between the web images and their auxiliary text terms, which can provide a good starting point for supporting automatic image-text alignment. Finally, a relevance re-ranking algorithm is developed to identify the most relevant visual text terms for the visually-similar web images in the same cluster. Our experiments on large-scale web images have obtained very positive results.

[1]  Xian-Sheng Hua,et al.  Finding image exemplars using fast sparse affinity propagation , 2008, ACM Multimedia.

[2]  Brendan J. Frey,et al.  Hierarchical Affinity Propagation , 2011, UAI.

[3]  F. Quimby.  What's in a picture? , 1993, Laboratory animal science.

[4]  Jianping Fan,et al.  An Interactive Approach for Filtering Out Junk Images From Keyword-Based Google Search Results , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Wei-Ying Ma,et al.  Hierarchical clustering of WWW image search results using visual, textual and link information , 2004, MULTIMEDIA '04.

[6]  Wei-Ying Ma,et al.  Multi-model similarity propagation and its application for web image retrieval , 2004, MULTIMEDIA '04.

[7]  Changhu Wang,et al.  Image annotation refinement using random walk with restarts , 2006, MM '06.

[8]  Jianping Fan,et al.  Leveraging loosely-tagged images and inter-object correlations for tag recommendation , 2010, ACM Multimedia.

[9]  R. Manmatha,et al.  Multiple Bernoulli relevance models for image and video annotation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[10]  Xian-Sheng Hua,et al.  Video search re-ranking via multi-graph propagation , 2007, ACM Multimedia.

[11]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[12]  Jianping Fan,et al.  Leveraging auxiliary text terms for automatic image annotation , 2011, WWW.

[13]  Marie-Francine Moens,et al.  Cross-Media Alignment of Names and Faces , 2010, IEEE Transactions on Multimedia.

[14]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[15]  Pietro Perona,et al.  Learning object categories from Google's image search , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Jianping Fan,et al.  Harvesting large-scale weakly-tagged image databases from the web , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Shih-Fu Chang,et al.  Video search reranking via information bottleneck principle , 2006, MM '06.

[18]  Dong Liu,et al.  Tag ranking , 2009, WWW '09.

[19]  Jianping Fan,et al.  Automatic image-text alignment for large-scale web image indexing and retrieval , 2015, Pattern Recognit..