Generating “Visual Clouds” from Multiplex Networks for TV News Archive Query Visualization

Advances in multimedia analysis, enables access and indexing of huge and rich video content. However, delivering this heterogeneous content remains a challenge. We proposes the use of multiplex networks to combine both textual and visual semantic cues extracted from TV news videos for interactive search refinement on heterogeneous data. After preprocessing, the indexed videos can be queried, for which result space is summarized in a visual cloud helping query refinement with contextual information by combining visual and textual cues. We leverage on properties of multiplex networks to extract a hierarchy of layers, which not only contextualizes results, but also guides the construction of our visual cloud, and its interactions.

[1]  Roberto Navigli,et al.  Clustering and Diversifying Web Search Results with Graph-Based Word Sense Induction , 2013, CL.

[2]  Guy Melançon,et al.  Entanglement in Multiplex Networks: Understanding Group Cohesion in Homophily Networks , 2014, Social Network Analysis.

[3]  Martin Wattenberg,et al.  Participatory Visualization with Wordle , 2009, IEEE Transactions on Visualization and Computer Graphics.

[4]  Shih-Fu Chang,et al.  News rover: exploring topical structures and serendipity in heterogeneous multimedia news , 2013, MM '13.

[5]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Daniel A. Keim,et al.  Incremental visual text analytics of news story development , 2012, Visualization and Data Analysis.

[7]  Andrea Marino,et al.  Topical clustering of search results , 2012, WSDM '12.

[8]  Ulrik Brandes,et al.  Organizing Search Results with a Reference Map , 2012, IEEE Transactions on Visualization and Computer Graphics.

[9]  James D. Foley,et al.  ResultMaps: Visualization for Search Interfaces , 2009, IEEE Transactions on Visualization and Computer Graphics.

[10]  K. Fujimura,et al.  BLOGRANGER – A Multi-faceted Blog Search Engine , 2006 .

[11]  Claudio Carpineto,et al.  A Survey of Automatic Query Expansion in Information Retrieval , 2012, CSUR.

[12]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[14]  Christian Biemann,et al.  Networks of Names: Visual Exploration and Semi‐Automatic Tagging of Social Networks from Newspaper Articles , 2014, Comput. Graph. Forum.

[15]  Andreas Noack,et al.  Modularity clustering is force-directed layout , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[17]  Frank Nack,et al.  [Invited Paper] Explain This to Me! , 2013 .

[18]  Shin'ichi Satoh,et al.  Image Flows Visualization for Inter-media Comparison , 2014, 2014 IEEE Pacific Visualization Symposium.

[19]  Duy-Dinh Le,et al.  Face Retrieval in Large-Scale News Video Datasets , 2013, IEICE Trans. Inf. Syst..

[20]  Tamara Munzner,et al.  A Multi-Level Typology of Abstract Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[21]  Mahadev Satyanarayanan,et al.  OpenFace: A general-purpose face recognition library with mobile applications , 2016 .

[22]  Shin'ichi Satoh,et al.  Topic Threading for Structuring a Large-Scale News Video Archive , 2004, CIVR.

[23]  Tamara Munzner,et al.  Detangler: Visual Analytics for Multiplex Networks , 2015, Comput. Graph. Forum.

[24]  Jianping Fan,et al.  Exploring Large-Scale Video News via Interactive Visualization , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.

[25]  Duy-Dinh Le,et al.  Indexing Faces in Broadcast News Video Archives , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[26]  George Karypis,et al.  Evaluation of hierarchical clustering algorithms for document datasets , 2002, CIKM '02.

[27]  Duy-Dinh Le,et al.  When face-tracking meets social networks: a story of politics in news videos , 2016, Applied Network Science.

[28]  Hongan Wang,et al.  Visualization of large hierarchical data by circle packing , 2006, CHI.

[29]  Michael Cardew-Hall,et al.  The folksonomy tag cloud: when is it useful? , 2008, J. Inf. Sci..

[30]  Kwan-Liu Ma,et al.  Semantic‐Preserving Word Clouds by Seam Carving , 2011, Comput. Graph. Forum.

[31]  Olivier Buisson,et al.  OTMedia: the French TransMedia news observatory , 2013, MM '13.

[32]  Hai Lin,et al.  Semantic word cloud generation based on word embeddings , 2016, 2016 IEEE Pacific Visualization Symposium (PacificVis).