Using contextual analysis for news event detection

The rapidly growing amount of newswire stories stored in electronic devices raises new challenges for information retrieval technology. Traditional query‐driven retrieval is not suitable for generic queries. It is desirable to have an intelligent system to automatically locate topically related events or topics in a continuous stream of newswire stories. This is the goal of automatic event detection. We propose a new approach to performing event detection from multilingual newswire stories. Unlike traditional methods which employ simple keyword matching, our method makes use of concept terms and named entities such as person, location, and organization names. Concept terms of a story are derived from statistical context analysis between sentences in the news story and stories in the concept database. We have conducted a set of experiments to study the effectiveness of our approach. The results show that the performance of detection using concept terms together with story keywords is better than traditional methods which only use keyword representation. © 2001 John Wiley & Sons, Inc.

[1]  Hsinchun Chen,et al.  Exploring the use of concept spaces to improve medical information retrieval , 2000, Decis. Support Syst..

[2]  W. Bruce Croft,et al.  Query expansion using local and global document analysis , 1996, SIGIR '96.

[3]  Yiming Yang,et al.  A study of retrospective and on-line event detection , 1998, SIGIR '98.

[4]  Yiming Yang,et al.  Topic Detection and Tracking Pilot Study Final Report , 1998 .

[5]  Tobun Dorbin Ng,et al.  Alleviating search uncertainty through concept associations: automatic indexing, co-occurrence analysis, and parallel computing , 1998 .

[6]  Satya Dharanipragada,et al.  Segmentation and Detection at IBM , 2002 .

[7]  Hsinchun Chen,et al.  A concept space approach to addressing the vocabulary problem in scientific information retrieval: an experiment on the worm community system , 1997 .

[8]  Hsinchun Chen,et al.  Internet Browsing and Searching: User Evaluations of Category Map and Concept Space Techniques , 1998, J. Am. Soc. Inf. Sci..

[9]  Helen M. Meng,et al.  An Analytical Study of Transformational Tagging for Chinese Text , 1999, ROCLING.

[10]  W. Bruce Croft,et al.  Text Segmentation by Topic , 1997, ECDL.

[11]  James Allan,et al.  On-Line New Event Detection and Tracking , 1998, SIGIR.

[12]  Hsinchun Chen,et al.  A Concept Space Approach to Addressing the Vocabulary Problem in Scientific Information Retrieval: An Experiment on the Worm Community System , 1997, J. Am. Soc. Inf. Sci..

[13]  Hsinchun Chen,et al.  Internet Browsing and Searching: User Evaluations of Category Map and Concept Space Techniques , 1998, J. Am. Soc. Inf. Sci..

[14]  Hsinchun Chen,et al.  Alleviating Search Uncertainty Through Concept Associations: Automatic Indexing, Co-Occurrence Analysis, and Parallel Computing , 1998, J. Am. Soc. Inf. Sci..

[15]  Jonathan Yamron,et al.  Statistical Models for Tracking and Detection , 2000 .

[16]  Vincent Kanade,et al.  Clustering Algorithms , 2021, Wireless RF Energy Transfer in the Massive IoT Era.

[17]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[18]  James Allan,et al.  Detections , Bounds , and Timelines : UMass and TDT-3 , 2000 .