A Classification Based Framework for Concept Summarization

In this paper we propose a novel classification based framework for finding a small number of images that summarize a given concept. Our method exploits metadata information available with the images to get category information using Latent Dirichlet Allocation. Using this category information for each image, we solve the underlying classification problem by building a sparse classifier model for each concept. We demonstrate that the images that specify the sparse model form a good summary. In particular, our summary satisfies important properties such as likelihood, diversity and balance in both visual and semantic sense. Furthermore, the framework allows users to specify desired distributions over categories to create personalized summaries.\eat{ We demonstrate the efficacy of our method on seven broad query types - sports, news, celebrities, events, travel, country and abstract.} Experimental results on seven broad query types show that the proposed method performs better than state-of-the-art methods.\eat{ in terms of satisfying important visual and semantic properties both qualitatively and quantitatively. We observe from editorial evaluation that around $78$\% of our summaries are of high enough quality to be shown directly to the web users with minimal or no modifications.

[1]  Jianping Fan,et al.  A novel approach to enable semantic and visual image summarization for exploratory image search , 2008, MIR '08.

[2]  Yong Yu,et al.  Enhancing diversity, coverage and balance for summarization through structure learning , 2009, WWW '09.

[3]  Mor Naaman,et al.  Generating diverse and representative image search results for landmarks , 2008, WWW.

[4]  Shih-Fu Chang,et al.  Video search reranking via information bottleneck principle , 2006, MM '06.

[5]  Alexander C. Berg,et al.  Finding iconic images , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Marc'Aurelio Ranzato,et al.  Unsupervised image ranking , 2009, LS-MMRM '09.

[7]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[9]  Ji Zhu,et al.  Kernel Logistic Regression and the Import Vector Machine , 2001, NIPS.

[10]  Jon M. Kleinberg,et al.  Mapping the world's photos , 2009, WWW '09.

[11]  Steven M. Seitz,et al.  Scene Summarization for Online Image Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.