论文信息 - Hierarchical topic detection in large digital news archives: Exploring a sample based approach

Hierarchical topic detection in large digital news archives: Exploring a sample based approach

Hierarchical topic detection is a new task in the TDT 2004 evaluation program, which aims to organize a collection of unstructured news data in a directed acyclic graph (DAG) structure, refecting the topics discussed in the collection, ranging from rather coarse category like nodes to file singular events. The HTD task poses interesting challenges since its evaluation metric is composed of a travel cost component refecting the time to fhd the node of interest starting from the top node and a quality cost component, determined by the quality of the selected node. We present a scalable architecture for HTD and compare several alternative choices for agglomerative clustering and DAG optimization in order to minimize the HTD cost metric. The alternatives are evaluated on the TDT3 and TDT5 test collections.

Dolf Trieschnigg | Wessel Kraaij | Wessel Kraaij | D. Trieschnigg

[1] David R. Karger,et al. Scatter/Gather: a cluster-based approach to browsing large document collections , 1992, SIGIR '92.

[2] C. J. van Rijsbergen,et al. Information Retrieval , 1979, Encyclopedia of GIS.

[3] Anil K. Jain,et al. Data clustering: a review , 1999, CSUR.

[4] James Allan,et al. Flexible intrinsic evaluation of hierarchical clustering for TDT , 2003, CIKM '03.

[5] Peter Willett,et al. Recent trends in hierarchic document clustering: A critical review , 1988, Inf. Process. Manag..

[6] James Allan,et al. Topic Detection and Tracking , 2002, The Information Retrieval Series.

[7] Wessel Kraaij,et al. Variations on language modeling for information retrieval , 2005, SIGF.

[8] George Karypis,et al. Evaluation of hierarchical clustering algorithms for document datasets , 2002, CIKM '02.

[9] Patrick Pantel,et al. Document clustering with committees , 2002, SIGIR '02.

[10] Wessel Kraaij,et al. Unsupervised Event Clustering in Multilingual News Streams , 2002 .

[11] Alan F. Smeaton,et al. An Architecture for Efficient Document Clustering and Retrieval on a Dynamic Collection of Newspaper Texts , 1998, BCS-IRSG Annual Colloquium on IR Research.