Hybrid image summarization

In this paper, we address a problem of managing tagged images with hybrid summarization. We formulate this problem as finding a few image exemplars to represent the image set semantically and visually and solve it in a hybrid way by exploiting both visual and textual information associated with images. We propose a novel approach, called Homogeneous and Heterogeneous Message Propagation (H2MP), which extends affinity propagation that only works over homogeneous relations to heterogeneous relations. The summary obtained by our approach is both visually and semantically satisfactory. The experimental results demonstrate the effectiveness and efficiency of the proposed approach.

[1]  Inderjit S. Dhillon,et al.  Co-clustering documents and words using bipartite spectral graph partitioning , 2001, KDD '01.

[2]  Inderjit S. Dhillon,et al.  Information-theoretic co-clustering , 2003, KDD '03.

[3]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[4]  Paul Clough,et al.  Automatically organising images using concept hierarchies , 2005 .

[5]  Andrew Blake,et al.  Digital tapestry [automatic image synthesis] , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Tao Qin,et al.  Web image clustering by consistent utilization of visual features and surrounding texts , 2005, MULTIMEDIA '05.

[7]  Mor Naaman,et al.  Generating summaries for large collections of geo-referenced photographs , 2006, WWW '06.

[8]  P. Schmitz,et al.  Inducing Ontology from Flickr Tags , 2006 .

[9]  Steven M. Seitz,et al.  Scene Summarization for Online Image Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[10]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[11]  Svetlana Lazebnik,et al.  Computing iconic summaries of general visual concepts , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Jing Hua,et al.  Graph theoretical framework for simultaneously integrating visual and textual features for efficient web image clustering , 2008, WWW.

[13]  Xian-Sheng Hua,et al.  Finding image exemplars using fast sparse affinity propagation , 2008, ACM Multimedia.

[14]  Harry Shum,et al.  Picture Collage , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Hao Xu,et al.  Summarizing tagged image collections by cross-media representativeness voting , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[16]  Xian-Sheng Hua,et al.  Interactive browsing via diversified visual summarization for image search results , 2011, Multimedia Systems.