Automatic generation of social media snippets for mobile browsing

The ongoing revolution in media consumption from traditional PCs to the pervasiveness of mobile devices is driving the adoption of social media in our daily lives. More and more people are using their mobile devices to enjoy social media content while on the move. However, mobile display constraints create challenges for presenting and authoring the rich media content on screens with limited display size. This paper presents an innovative system to automatically generate magazine-like social media visual summaries, which is called "snippet," for efficient mobile browsing. The system excerpts the most salient and dominant elements, i.e., a major picture element and a set of textual elements, from the original media content, and composes these elements into a text overlaid image by maximizing information perception. In particular, we investigate a set of aesthetic rules and visual perception principles to optimize the layout of the extracted elements by considering display constraints. As a result, browsing the snippet on mobile devices is just like quickly glancing at a magazine. To the best of our knowledge, this paper represents one of the first attempts at automatic social media snippet generation by studying aesthetic rules and visual perception principles. We have conducted experiments and user studies with social posts from news entities. We demonstrated that the generated snippets are effective at representing media content in a visually appealing and compact way, leading to a better user experience when consuming social media content on mobile devices.

[1]  J. Itten.  The art of color : the subjective experience and objective rationale of color , 1973 .

[2]  O. Sorkine,et al.  Color harmonization , 2006, SIGGRAPH 2006.

[3]  W. Chu.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2013 .

[4]  Yili Liu,et al.  Computational models and experimental investigations of effects of balance and symmetry on the aesthetics of text-overlaid images , 2010, Int. J. Hum. Comput. Stud..

[5]  Pirkko Oittinen,et al.  Semi-automated magazine layout using content-based image features , 2012, ACM Multimedia.

[6]  Saul Greenberg,et al.  How People Recognise Previously Seen Web Pages from Titles, URLs and Thumbnails , 2001 .

[7]  Susanne Boll,et al.  Automatic creation of photo books from stories in social media , 2011, TOMCCAP.

[8]  HongJiang Zhang,et al.  Text Area Detection from Video Frames , 2001, IEEE Pacific Rim Conference on Multimedia.

[9]  Tao Mei,et al.  Contextual Internet Multimedia Advertising , 2010, Proceedings of the IEEE.

[10]  Jaime Teevan,et al.  Visual snippets: summarizing web pages for search and revisitation , 2009, CHI.

[11]  Michael S. Bernstein,et al.  Eddi: interactive topic-based browsing of social status streams , 2010, UIST.

[12]  Yili Liu,et al.  Computational modeling and experimental investigation of effects of compositional elements on interface and design aesthetics , 2006, Int. J. Hum. Comput. Stud..

[13]  Hanan Samet,et al.  TwitterStand: news in tweets , 2009, GIS.

[14]  Harry Shum,et al.  Statistical Learning of Multi-view Face Detection , 2002, ECCV.

[15]  Tao Mei,et al.  Contextual in-image advertising , 2008, ACM Multimedia.

[16]  Rik Pieters,et al.  Attention Capture and Transfer in Advertising: Brand, Pictorial, and Text-Size Effects , 2004 .

[17]  Feng Wu,et al.  Visual summarization of web pages , 2010, SIGIR '10.

[18]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Joachim Meyer,et al.  Economic and subjective measures of the perceived value of aesthetics and usability , 2006, TCHI.

[20]  Hao Chen,et al.  Analyzing social media via event facets , 2012, ACM Multimedia.

[21]  Yunfei Chen,et al.  Evaluating the visual quality of web pages using a computational aesthetic approach , 2011, WSDM '11.

[22]  Shuming Shi,et al.  Improving relevance judgment of web search results with image excerpts , 2008, WWW.

[23]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[24]  Jeffrey P. Bigham,et al.  Multimodal summarization of complex sentences , 2011, IUI '11.