Towards hierarchical context: unfolding visual community potential for interactive video retrieval

Community structure as an interesting property of network has attracted wide attention from many research fields. In this paper, we exploit the visual community structure in visual-temporal correlation network and utilize it to improve interactive video retrieval. Firstly, we propose a hierarchical community-based feedback algorithm. By re-ranking the video shots through diffusion processes respectively on the inter-community and intra-community level, the feedback algorithm can make full use of the limited user feedback. Furthermore, since it avoids entire graph computation, the feedback algorithm can make quick responses to user feedback, which is particularly important for the large video collections. Secondly, we propose a community-based visualization interface called VideoMap. By organizing the video shots following the community structure, the VideoMap presents a comprehensive and informative view of the whole dataset to facilitate users’ access. Moreover, the VideoMap can help users to quickly locate the potential relevant regions and make active annotation according to the distribution of labeled samples on the VideoMap. Experiments on TRECVID 2009 search dataset demonstrate the efficiency of the feedback algorithm and the effectiveness of the visualization interface.

[1]  Matthieu Latapy,et al.  Computing Communities in Large Networks Using Random Walks , 2004, J. Graph Algorithms Appl..

[2]  Meng Wang,et al.  Video annotation by graph-based learning with neighborhood similarity , 2007, ACM Multimedia.

[3]  Reinhard Lipowsky,et al.  Network Brownian Motion: A New Method to Measure Vertex-Vertex Proximity and to Identify Communities and Subcommunities , 2004, International Conference on Computational Science.

[4]  Jean-Pierre Eckmann,et al.  Curvature of co-links uncovers hidden thematic layers in the World Wide Web , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Marcel Worring,et al.  The Mediamill Semantic Video Search Engine , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[6]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[7]  C. Lee Giles,et al.  Self-Organization and Identification of Web Communities , 2002, Computer.

[8]  Gene H. Golub,et al.  Extrapolation methods for accelerating PageRank computations , 2003, WWW '03.

[9]  M. Newman,et al.  Hierarchical structure and the prediction of missing links in networks , 2008, Nature.

[10]  João Magalhães,et al.  Video Retrieval Using Search and Browsing , 2004, TRECVID.

[11]  Timo Ojala,et al.  Cluster-temporal browsing of large news video databases , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[12]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[13]  Jingrui He,et al.  Manifold-ranking based image retrieval , 2004, MULTIMEDIA '04.

[14]  Rong Yan,et al.  Extreme video retrieval: joint maximization of human and computer performance , 2006, MM '06.

[15]  Christos Faloutsos,et al.  GCap: Graph-based Automatic Image Captioning , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[16]  T. Vicsek,et al.  Uncovering the overlapping community structure of complex networks in nature and society , 2005, Nature.

[17]  Shih-Fu Chang,et al.  Video search reranking through random walk over document-level context graph , 2007, ACM Multimedia.

[18]  Brian Kernighan,et al.  An efficient heuristic for partitioning graphs , 1970 .

[19]  Jiangchuan Liu,et al.  Statistics and Social Network of YouTube Videos , 2008, 2008 16th Interntional Workshop on Quality of Service.

[20]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[21]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Kristina Lerman,et al.  Community Detection Using a Measure of Global Influence , 2008, SNAKDD.

[23]  Yongdong Zhang,et al.  VideoMap: an interactive video retrieval system of MCG-ICT-CAS , 2009, CIVR '09.

[24]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[25]  C. Lee Giles,et al.  Efficient identification of Web communities , 2000, KDD '00.

[26]  M. Newman,et al.  Finding community structure in networks using the eigenvectors of matrices. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  Yiannis Kompatsiaris,et al.  A Graph-Based Clustering Scheme for Identifying Related Tags in Folksonomies , 2010, DaWak.

[28]  A-L Barabási,et al.  Structure and tie strengths in mobile communication networks , 2006, Proceedings of the National Academy of Sciences.

[29]  Andreas Argyriou.  Efficient Approximation Methods for Harmonic Semi-Supervised Learning , 2022 .

[30]  Jure Leskovec,et al.  Community Structure in Large Networks: Natural Cluster Sizes and the Absence of Large Well-Defined Clusters , 2008, Internet Math..

[31]  Mason A. Porter,et al.  Communities in Networks , 2009, ArXiv.

[32]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[33]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[34]  Albert-László Barabási,et al.  Understanding individual human mobility patterns , 2008, Nature.

[35]  M E J Newman,et al.  Finding and evaluating community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[36]  Padhraic Smyth,et al.  A Spectral Clustering Approach To Finding Communities in Graph , 2005, SDM.

[37]  Chong-Wah Ngo,et al.  Towards optimal bag-of-features for object categorization and semantic video retrieval , 2007, CIVR '07.

[38]  Yong-Yeol Ahn,et al.  Communities and Hierarchical Organization of Links in Complex Networks , 2009 .

[39]  A. Barabasi,et al.  Quantifying social group evolution , 2007, Nature.

[40]  Thomas S. Huang,et al.  One-class SVM for learning in image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[41]  Yongdong Zhang,et al.  Distribution-based concept selection for concept-based video retrieval , 2009, ACM Multimedia.

[42]  Yiannis Kompatsiaris,et al.  Image clustering through community detection on hybrid image similarity graphs , 2010, 2010 IEEE International Conference on Image Processing.

[43]  Brian W. Kernighan,et al.  An efficient heuristic procedure for partitioning graphs , 1970, Bell Syst. Tech. J..

[44]  Wei-Ying Ma,et al.  Learning similarity measure for natural image retrieval with relevance feedback , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[45]  Malik Yousef,et al.  One-Class SVMs for Document Classification , 2002, J. Mach. Learn. Res..

[46]  Marcel Worring,et al.  MediaMill: exploring news video archives based on learned semantics , 2005, MULTIMEDIA '05.

[47]  M. Newman,et al.  Finding community structure in very large networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[48]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..