News Event Understanding by Mining Latent Factors From Multimodal Tensors

We present a novel and efficient constrained tensor factorization algorithm that first represents a video archive, of multimedia news stories concerning a news event, as a sparse tensor of order 4. The dimensions correspond to extracted visual memes, verbal tags, time periods and cultures. The iterative algorithm then approximately but accurately ex- tracts coherent quad-clusters, each of which represents a significant summary of an important independent aspect of the news event. We give examples of quad-clusters extracted from tensors with at least 108 entries derived from the international news coverage of the Ebola epidemic, AirAsia flight Q8501 and Zika virus. We show the method is fast, can be tuned to give preferences to any subset of its four dimensions, and exceeds three existing methods in performance.

[1]  Yu Zong,et al.  Web Co-clustering of Usage Network Using Tensor Decomposition , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[2]  Dong Liu,et al.  A Bayesian Approach to Multimodal Visual Dictionary Learning , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Bin Ma,et al.  Broadcast News Story Segmentation Using Conditional Random Fields and Multimodal Features , 2012, IEICE Trans. Inf. Syst..

[4]  Lie Lu,et al.  Co-clustering for Auditory Scene Categorization , 2008, IEEE Transactions on Multimedia.

[5]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Yale Song,et al.  Video co-summarization: Video summarization by visual co-occurrence , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Shih-Fu Chang,et al.  Structured exploration of who, what, when, and where in heterogeneous multimedia news sources , 2013, ACM Multimedia.

[8]  Chong-Wah Ngo,et al.  Measuring novelty and redundancy with multiple modalities in cross-lingual broadcast news , 2008, Comput. Vis. Image Underst..

[9]  Alberto Messina,et al.  Hyper Media News: a fully automated platform for large scale analysis, production and distribution of multimodal news content , 2011, Multimedia Tools and Applications.

[10]  Inderjit S. Dhillon,et al.  Co-clustering documents and words using bipartite spectral graph partitioning , 2001, KDD '01.

[11]  Arindam Banerjee,et al.  Multi-way Clustering on Relation Graphs , 2007, SDM.

[12]  John Skvoretz,et al.  Node centrality in weighted networks: Generalizing degree and shortest paths , 2010, Soc. Networks.

[13]  Jure Leskovec,et al.  Overlapping community detection at scale: a nonnegative matrix factorization approach , 2013, WSDM.

[14]  Inderjit S. Dhillon,et al.  A generalized maximum entropy approach to bregman co-clustering and matrix approximation , 2004, J. Mach. Learn. Res..

[15]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[16]  Georges Quénot,et al.  Automatic Story Segmentation for TV News Video Using Multiple Modalities , 2012, Int. J. Digit. Multim. Broadcast..

[17]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[18]  Chong-Wah Ngo,et al.  Threading and Autodocumenting News Videos , 2006 .

[19]  Arindam Banerjee,et al.  Bayesian Co-clustering , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[20]  Christos Faloutsos,et al.  A General Suspiciousness Metric for Dense Blocks in Multimodal Data , 2015, 2015 IEEE International Conference on Data Mining.

[21]  Slim Essid,et al.  Smooth Nonnegative Matrix Factorization for Unsupervised Audiovisual Document Structuring , 2013, IEEE Transactions on Multimedia.

[22]  Munmun De Choudhury,et al.  Discovering multirelational structure in social media streams , 2012, TOMCCAP.

[23]  John R. Kender,et al.  An adaptive anchor frame detection algorithm based on background detection for news video analysis , 2016, 2016 International Conference on Audio, Language and Image Processing (ICALIP).

[24]  Inderjit S. Dhillon,et al.  Information-theoretic co-clustering , 2003, KDD '03.

[25]  Nikos D. Sidiropoulos,et al.  From K-Means to Higher-Way Co-Clustering: Multilinear Decomposition With Sparse Latent Factors , 2013, IEEE Transactions on Signal Processing.