Generation of Multimedia Artifacts: An Extractive Summarization-based Approach

We explore methods for content selection and address the issue of coherence in the context of the generation of multimedia artifacts. We use audio and video to present two case studies: generation of film tributes, and lecture-driven science talks. For content selection, we use centrality-based and diversity-based summarization, along with topic analysis. To establish coherence, we use the emotional content of music, for film tributes, and ensure topic similarity between lectures and documentaries, for science talks. Composition techniques for the production of multimedia artifacts are addressed as a means of organizing content, in order to improve coherence. We discuss our results considering the above aspects.

[1]  Edward Branigan.  Narrative Comprehension and Film , 1992 .

[2]  Florian Metze,et al.  Beyond audio and video retrieval: towards multimedia summarization , 2012, ICMR.

[3]  Yu Qiao,et al.  Automatic music video generation: cross matching of music and image , 2012, ACM Multimedia.

[4]  Patrick F. Reidy.  An Introduction to Latent Semantic Analysis , 2009 .

[5]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[6]  Ricardo Ribeiro,et al.  Revisiting Centrality-as-Relevance: Support Sets and Similarity as Geometric Proximity: Extended abstract , 2013, IJCAI.

[7]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[8]  Dragomir R. Radev,et al.  LexRank: Graph-based Lexical Centrality as Salience in Text Summarization , 2004, J. Artif. Intell. Res..

[9]  Björn W. Schuller,et al.  Real-life voice activity detection with LSTM Recurrent Neural Networks and an application to Hollywood movies , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[11]  Xiaojin Zhu,et al.  Improving Diversity in Ranking using Absorbing Random Walks , 2007, NAACL.

[12]  Thierry Bertin-Mahieux,et al.  The Million Song Dataset , 2011, ISMIR.

[13]  Coskun Bayrak,et al.  Sports video summarization based on motion analysis , 2013, Comput. Electr. Eng..

[14]  Patrick Bouthemy,et al.  Unsupervised soccer video abstraction based on pitch, dominant color and camera motion analysis , 2004, MULTIMEDIA '04.

[15]  Peter W. Foltz,et al.  The Measurement of Textual Coherence with Latent Semantic Analysis. , 1998 .

[16]  Björn Schuller,et al.  Opensmile: the munich versatile and fast open-source audio feature extractor , 2010, ACM Multimedia.

[17]  Lynda Hardman,et al.  Automatic generation of matter-of-opinion video documentaries , 2008, J. Web Semant..

[18]  Petros Maragos,et al.  Multimodal Saliency and Fusion for Movie Summarization Based on Aural, Visual, and Textual Attention , 2013, IEEE Transactions on Multimedia.

[19]  Chris D. Paice,et al.  Constructing literature abstracts by computer: Techniques and prospects , 1990, Inf. Process. Manag..

[20]  Lie Lu,et al.  Automatic music video generation based on temporal pattern analysis , 2004, MULTIMEDIA '04.

[21]  Otthein Herzog,et al.  Automatic Generation of Movie Trailers using Ontologies , 2007 .

[22]  Patrick Bouthemy,et al.  Tennis video abstraction from audio and visual cues , 2004, IEEE 6th Workshop on Multimedia Signal Processing, 2004..

[23]  Kiyoharu Aizawa,et al.  Automatic trailer generation , 2010, ACM Multimedia.

[24]  Thomas D. C. Little,et al.  Automatic Composition Techniques for Video Production , 1998, IEEE Trans. Knowl. Data Eng..

[25]  Xin Liu,et al.  Generic text summarization using relevance measure and latent semantic analysis , 2001, SIGIR '01.

[26]  Qingzhong Liu,et al.  Scalable Secure MJPEG Video Streaming , 2012, 2012 26th International Conference on Advanced Information Networking and Applications Workshops.

[27]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[28]  Bernard Mérialdo,et al.  Multi-video summarization based on Video-MMR , 2010, 11th International Workshop on Image Analysis for Multimedia Interactive Services WIAMIS 10.

[29]  Oriol Nieto,et al.  Music segment similarity using 2D-Fourier Magnitude Coefficients , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).