Rethinking Summarization and Storytelling for Modern Social Multimedia

Traditional summarization initiatives have been focused on specific types of documents such as articles, reviews, videos, image feeds, or tweets, a practice which may result in pigeonholing the summarization task in the context of modern, content-rich multimedia collections. Consequently, much of the research to date has revolved around mostly toy problems in narrow domains and working on single-source media types. We argue that summarization and story generation systems need to refocus the problem space in order to meet the information needs in the age of user-generated content in different formats and languages. Here we create a framework for flexible multimedia storytelling. Narratives, stories, and summaries carry a set of challenges in big data and dynamic multi-source media that give rise to new research in spatial-temporal representation, viewpoint generation, and explanation.

[1]  Paul Over,et al.  The DUC summarization evaluations , 2002 .

[2]  Alan F. Smeaton,et al.  LifeLogging: Personal Big Data , 2014, Found. Trends Inf. Retr..

[3]  Pablo Rodriguez,et al.  I tube, you tube, everybody tubes: analyzing the world's largest user generated content video system , 2007, IMC '07.

[4]  Mor Naaman,et al.  Less talk, more rock: automated organization of community-contributed collections of concert videos , 2009, WWW '09.

[5]  Dragomir R. Radev,et al.  Introduction to the Special Issue on Summarization , 2002, CL.

[6]  Masataka Goto,et al.  A chorus section detection method for musical audio signals and its application to a music listening station , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[7]  Gautam Shroff,et al.  Acquiring competitive intelligence from social media , 2011, MOCR_AND '11.

[8]  Jiayu Tang,et al.  What Else Is There? Search Diversity Examined , 2009, ECIR.

[9]  Steve E Hodges,et al.  Wearable cameras in health: the state of the art and future possibilities. , 2013, American journal of preventive medicine.

[10]  Kristen Grauman,et al.  Story-Driven Summarization for Egocentric Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  David A. Shamma,et al.  Tweet the debates: understanding community annotation of uncollected sources , 2009, WSM@MM.

[12]  Stevan Rudinac,et al.  Learning Crowdsourced User Preferences for Visual Summarization of Image Collections , 2013, IEEE Transactions on Multimedia.

[13]  Shingo Uchihashi,et al.  Video Manga: generating semantically meaningful video summaries , 1999, MULTIMEDIA '99.

[14]  Jade Goldstein-Stewart,et al.  The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries , 1998, SIGIR Forum.

[15]  Mark Sanderson,et al.  Diversity in Photo Retrieval: Overview of the ImageCLEFPhoto Task 2009 , 2009, CLEF.

[16]  Yang Yang,et al.  Multimedia Summarization for Social Events in Microblog Stream , 2015, IEEE Transactions on Multimedia.

[17]  Yi Yang,et al.  Harry Potter's Marauder's Map: Localizing and Tracking Multiple Persons-of-Interest by Nonnegative Discretization , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Wei Chai,et al.  Semantic Segmentation and Summarization of Music , 2006 .

[19]  Bernard Mérialdo,et al.  VERT: automatic evaluation of video summaries , 2010, ACM Multimedia.

[20]  Alan F. Smeaton,et al.  Constructing a SenseCam visual diary as a media process , 2008, Multimedia Systems.

[21]  Jiang-Ming Yang,et al.  Generating location overviews with images and tags by mining user-generated travelogues , 2009, ACM Multimedia.

[22]  M. de Rijke,et al.  Personalized time-aware tweets summarization , 2013, SIGIR.

[23]  Ani Nenkova,et al.  Evaluating Content Selection in Summarization: The Pyramid Method , 2004, NAACL.

[24]  Hosung Park,et al.  What is Twitter, a social network or a news media? , 2010, WWW '10.

[25]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[26]  Paul Over,et al.  The trecvid 2008 BBC rushes summarization evaluation , 2008, TVS '08.

[27]  Wei Chai,et al.  Semantic segmentation and summarization of music: methods based on tonality and recurrent structure , 2006, IEEE Signal Processing Magazine.

[28]  Ani Nenkova,et al.  A Survey of Text Summarization Techniques , 2012, Mining Text Data.

[29]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[30]  Shih-Fu Chang,et al.  A utility framework for the automatic generation of audio-visual skims , 2002, MULTIMEDIA '02.

[31]  Jaideep Srivastava,et al.  Social Multimedia Computing , 2010, Computer.

[32]  Takeo Kanade,et al.  Intelligent Access to Digital Video: Informedia Project , 1996, Computer.

[33]  Yong Jae Lee,et al.  Discovering important people and objects for egocentric video summarization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Ximena Olivares,et al.  Visual diversification of image search results , 2009, WWW '09.

[35]  Marcel Worring,et al.  What Multimedia Sentiment Analysis Says About City Liveability , 2016, ECIR.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Mark B. Sandler,et al.  Towards Music Structural Segmentation across Genres , 2017, ACM Trans. Intell. Syst. Technol..

[38]  Johanna D. Moore,et al.  Twitter Sentiment Analysis: The Good the Bad and the OMG! , 2011, ICWSM.

[39]  Licia Capra,et al.  Urban Computing: Concepts, Methodologies, and Applications , 2014, TIST.

[40]  Henning Müller,et al.  Result diversification in social image retrieval: a benchmarking framework , 2014, Multimedia Tools and Applications.

[41]  Hoa Trang Dang,et al.  Overview of the TAC 2008 Update Summarization Task , 2008, TAC.

[42]  Panagiotis Takis Metaxas,et al.  The power of prediction with social media , 2013, Internet Res..

[43]  David A. Shamma,et al.  YFCC100M , 2015, Commun. ACM.

[44]  Stevan Rudinac,et al.  Generating Visual Summaries of Geographic Areas Using Community-Contributed Images , 2013, IEEE Transactions on Multimedia.

[45]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[46]  Mor Naaman,et al.  Generating diverse and representative image search results for landmarks , 2008, WWW.

[47]  Hoa Trang Dang,et al.  Overview of DUC 2006 , 2006 .

[48]  Hoa Trang Dang,et al.  Overview of DUC 2005 , 2005 .

[49]  Hoa Trang Dang,et al.  DUC 2005: Evaluation of Question-Focused Summarization Systems , 2006 .

[50]  John R. Kender,et al.  Visual memes in social media: tracking real-world news in YouTube videos , 2011, ACM Multimedia.

[51]  Hung-Khoon Tan,et al.  Beyond search: Event-driven summarization for web videos , 2011, TOMCCAP.

[52]  Markus Schedl,et al.  The neglected user in music information retrieval research , 2013, Journal of Intelligent Information Systems.

[53]  Lexing Xie,et al.  Event Mining in Multimedia Streams , 2008, Proceedings of the IEEE.