Automatic suggestion of presentation image for storytelling

Digital storytelling applications are playing an increasingly important role in people's daily life. In contemporary storytelling applications such as PowerPoint presentation and macro/micro blogs, good presentation images are always highly desired by content creators to boost their presentation in an intuitive and attractive way. Existing studies, however, have not yet addressed the challenging problem of how to select the most appropriate presentation images for storytelling. In this paper, we formulate this problem of presentation image suggestion (given a textual query) as selecting images by maximizing visual and semantic diversity from web image search results of suggested queries. The proposed framework consists of two novel components: 1) click-through-based query suggestion, which is designed to suggest textual queries by searching relevant queries in a constructed query graph that can reflect diverse aspects of a given query, and 2) query-based image selection, which selects the most appropriate presentation images by keeping semantic relevance while maximizing visual diversity and quality, using a novel model based on Conditional Random Field (CRF) by individual and correlation characters. We evaluate the proposed approach by comparing with several baselines and a thorough subjective survey. The evaluations show inspiring results using the proposed approach for automatic suggestion of images for storytelling.

[1]  Réka Albert,et al.  Near linear time algorithm to detect community structures in large-scale networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  Tao Mei,et al.  Community Discovery from Movie and Its Application to Poster Generation , 2011, MMM.

[3]  Yong Wang,et al.  Coherent image annotation by learning semantic distance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Meng Wang,et al.  Visual query suggestion , 2009, ACM Multimedia.

[5]  Qi Tian,et al.  Multimedia search reranking: A literature survey , 2014, CSUR.

[6]  Henning Müller,et al.  Benchmarking result diversification in social image retrieval , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[7]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  Tao Mei,et al.  Automatic generation of social media snippets for mobile browsing , 2013, ACM Multimedia.

[9]  Jing Wang,et al.  Clickage: towards bridging semantic and intent gaps via mining click logs of search engines , 2013, ACM Multimedia.

[10]  Claudio Carpineto,et al.  A Survey of Automatic Query Expansion in Information Retrieval , 2012, CSUR.

[11]  Tao Mei,et al.  Learning to personalize trending image search suggestion , 2014, SIGIR.

[12]  Yi Yang,et al.  DevNet: A Deep Event Network for multimedia event detection and evidence recounting , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Mark Sanderson,et al.  Diversity in Photo Retrieval: Overview of the ImageCLEFPhoto Task 2009 , 2009, CLEF.

[14]  Tao Mei,et al.  Automatic Generation of Visual-Textual Presentation Layout , 2016, ACM Trans. Multim. Comput. Commun. Appl..