The use of topic evolution to help users browse and find answers in news video corpus

Earlier research in news video has been focusing mainly on improving retrieval accuracies given the limited amount of extractable video semantics. In this paper, we propose an enhancement to news video searching by leveraging extractable video semantics coupled with relevant external information resources to support event-based analysis; leading to discovery of topic hierarchy for browsing key events and supporting question answering (QA). We introduce topic browsing based on news structures obtained through hierarchical clustering and threading, with emphasis on interesting events determined by measuring the amount of "web activities" on these events on Blog sites. For QA, we employ extensive query analysis to obtain various query features in addition to the topic hierarchical structures to answer both context-oriented and visual-oriented questions. Our main contributions includes: (a) combining multimodal event information extracted from news video, web news articles and news blogs to support event analysis, (b) introducing topic evolution browsing based on users' interest and (c) extending QA on top of topic hierarchy to handle various types of specialized video queries. Experiments performed on 70 hours of multilingual news from TRECVID 2005 dataset shows that the proposed approach is effective and appealing to users.

[1]  Shih-Fu Chang,et al.  COLUMBIA-IBM NEWS VIDEO STORY SEGMENTATION IN TRECVID 2004 , 2005 .

[2]  Grace Hui Yang,et al.  Structured use of external knowledge for event-based open domain question answering , 2003, SIGIR.

[3]  Jin Zhao,et al.  Video Retrieval Using High Level Features: Exploiting Query Matching and Confidence-Based Weighting , 2006, CIVR.

[4]  Tat-Seng Chua,et al.  TRECVID 2005 by NUS PRIS , 2005, TRECVID.

[5]  Qi Tian,et al.  News video search with fuzzy event clustering using high-level features , 2006, MM '06.

[6]  Rong Yan,et al.  Learning query-class dependent weights in automatic video retrieval , 2004, MULTIMEDIA '04.

[7]  Tobun Dorbin Ng,et al.  Collages as dynamic summaries for news video , 2002, MULTIMEDIA '02.

[8]  Harry Bruce,et al.  Perceptions of the Internet: what people think when they search the Internet for information , 1999, Internet Res..

[9]  W. Bruce Croft,et al.  The use of phrases and structured queries in information retrieval , 1991, SIGIR '91.

[10]  James Allan,et al.  On-Line New Event Detection and Tracking , 1998, SIGIR.

[11]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[12]  Gisbert Schneider,et al.  A Hierarchical Clustering Approach for Large Compound Libraries , 2005, J. Chem. Inf. Model..

[13]  Christian Petersohn.  Fraunhofer HHI at TRECVID 2004: Shot Boundary Detection System , 2004, TRECVID.

[14]  Howard D. Wactlar,et al.  Complementary video and audio analysis for broadcast news archives , 2000, CACM.

[15]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[16]  Anthony R. Davis,et al.  Retrieving what's relevant in audio and video: statistics and linguistics in combination , 2004, RIAO.

[17]  Grace Hui Yang,et al.  VideoQA: question answering on news video , 2003, MULTIMEDIA '03.

[18]  John R. Smith,et al.  IBM Research TRECVID-2009 Video Retrieval System , 2009, TRECVID.

[19]  Philip Resnik,et al.  Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural Language , 1999, J. Artif. Intell. Res..

[20]  Tat-Seng Chua,et al.  Multi-faceted contextual model for person identification in news video , 2006, 2006 12th International Multi-Media Modelling Conference.

[21]  Sariel Har-Peled,et al.  How Fast Is the k-Means Method? , 2005, SODA '05.

[22]  Martin Franz,et al.  Influence of speech recognition errors on topic detection (poster session) , 2000, SIGIR '00.

[23]  John Adcock,et al.  Interactive Video Search Using Multilevel Indexing , 2005, CIVR.

[24]  Chin-Hui Lee,et al.  The segmentation of news video into story units , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.