Evaluation of an interactive topic detection and tracking interface

Interactive Topic Detection and Tracking (iTDT) refers to the TDT works which focus on user interaction, user evaluation and user interfaces aspects. This article investigates and identifies elements of the design of an interface that aims to facilitate journalists performing TDT tasks such as tracking and detection. It presents an (iTDT) interface called Interactive Event Tracking (iEvent), and evaluates the usability of the features introduced. The findings indicate the features that facilitated the participants in performing both tasks: cluster labelling and top terms features in Cluster View, a histogram with the timeline and document content features in Document View, and a keyword approach feature in Term View. Meanwhile, features such as cluster visualisation in Cluster View and histogram with the timeline in Term View only facilitated participants during the tracking task. The study shows that the interface enables journalists to perform well in TDT tasks.

[1]  Fabio Crestani,et al.  A Comparison of Named Entity Patterns from a User Analysis and a System Analysis , 2008, ECIR.

[2]  Marti A. Hearst.  TileBars: visualization of term distribution information in full text information access , 1995, CHI '95.

[3]  Ralph Grishman,et al.  Message Understanding Conference- 6: A Brief History , 1996, COLING.

[4]  Peter Ingwersen,et al.  The Turn - Integration of Information Seeking and Retrieval in Context , 2005, The Kluwer International Series on Information Retrieval.

[5]  Earl R. Babbie,et al.  The Basics Of Social Research , 1998 .

[6]  Ben Shneiderman,et al.  Network Visualization by Semantic Substrates , 2006, IEEE Transactions on Visualization and Computer Graphics.

[7]  James Allan,et al.  Automatic generation of overview timelines , 2000, SIGIR '00.

[8]  Kalina Bontcheva,et al.  GATE: an Architecture for Development of Robust HLT applications , 2002, ACL.

[9]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[10]  Mark Liberman,et al.  Corpora for topic detection and tracking , 2002 .

[11]  J. Teevan.  The ReSearch Engine : Helping People Return to Information in Dynamic Information Environments , 2022 .

[12]  Dragomir R. Radev,et al.  Using Random Walks for Question-focused Sentence Retrieval , 2005, HLT.

[13]  Masnizah Mohd.  Named entity patterns across news domains , 2007 .

[14]  Fabio Crestani,et al.  Context: Nature, Impact, and Role - 5th International Conference on Conceptions of Library and Information Sciences, CoLIS 2005, Glasgow, UK, June 4-8, 2005, Proceedings , 2005, CoLIS.

[15]  Yiming Yang,et al.  Topic Detection and Tracking Pilot Study Final Report , 1998 .

[16]  Jonathan G. Fiscus,et al.  Topic detection and tracking evaluation overview , 2002 .

[17]  Graham J. Wills,et al.  High interaction graphics , 1995 .

[18]  S. Davies,et al.  Big brother : Britain's web of surveillance and the new technological order , 1996 .

[19]  Pia Borlund,et al.  Evaluation of interactive information retrieval systems , 2000 .

[20]  James Allan,et al.  Lighthouse: showing the way to relevant information , 2000, IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings.

[21]  Fabio Crestani,et al.  Recasting the context in information retrieval , 1996 .

[22]  C. Osgood,et al.  The Measurement of Meaning , 1958 .

[23]  Padmini Srinivasan,et al.  A cluster-based approach to broadcast news , 2002 .

[24]  Benjamin B. Bederson,et al.  PhotoMesa: a zoomable image browser using quantum treemaps and bubblemaps , 2001, UIST '01.

[25]  Fabio Crestani,et al.  Towards personalized distributed information retrieval , 2008, SIGIR '08.

[26]  Rafael Berlanga Llavori,et al.  JERARTOP: A New Topic Detection System , 2004, CIARP.

[27]  Hsin-Hsi Chen,et al.  An NLP & IR approach to topic detection , 2002 .

[28]  Hitoshi Isahara,et al.  IREX: IR & IE Evaluation Project in Japanese , 2000, LREC.

[29]  James Allan,et al.  Interactive Cluster Visualization for Information Retrieval , 1997 .

[30]  James Allan,et al.  Detection As Multi-Topic Tracking , 2002, Information Retrieval.

[31]  Ramesh Nallapati,et al.  Event threading within news topics , 2004, CIKM '04.

[32]  Gareth J. F. Jones,et al.  A visualisation tool for topic tracking analysis and development , 2002, SIGIR '02.

[33]  Marti A. Hearst.  Search User Interfaces , 2009 .

[34]  Teun A. van Dijk,et al.  Discourse Analysis: Its Development and Application to the Structure of News , 1983 .

[35]  Gerald Kowalski,et al.  Information Retrieval Systems: Theory and Implementation , 1997 .

[36]  Ben Shneiderman,et al.  The Future of Graphic User Interfaces: Personal Role Managers , 1994, BCS HCI.

[37]  James Allan,et al.  Introduction to topic detection and tracking , 2002 .

[38]  Thorsten Brants,et al.  A System for new event detection , 2003, SIGIR.

[39]  Donna K. Harman,et al.  Overview of the Eighth Text REtrieval Conference (TREC-8) , 1999, TREC.

[40]  Yannis Avrithis,et al.  Personalized information retrieval in context , 2006 .

[41]  James D. Hollan,et al.  Pad++: A Zoomable Graphical Sketchpad For Exploring Alternate Interface Physics , 1996, J. Vis. Lang. Comput..

[42]  Ryen W. White,et al.  An implicit feedback approach for interactive information retrieval , 2006, Inf. Process. Manag..

[43]  Yiming Yang,et al.  Topic-conditioned novelty detection , 2002, KDD.

[44]  James Allan,et al.  Temporal summaries of new topics , 2001, SIGIR '01.

[45]  Tom Rosenstiel,et al.  The Elements of Journalism: What Newspeople Should Know and the Public Should Expect , 2001 .

[46]  Fabio Crestani,et al.  Design of an Interface for Interactive Topic Detection and Tracking , 2009, FQAS.

[47]  Claudia Niederée,et al.  Entity Name System: The Back-Bone of an Open and Scalable Web of Data , 2008, 2008 IEEE International Conference on Semantic Computing.

[48]  Joseph Becker,et al.  Information Retrieval and Processing , 1975 .

[49]  Pia Borlund,et al.  Experimental components for the evaluation of interactive information retrieval systems , 2000, J. Documentation.

[50]  Grady Booch,et al.  Object-Oriented Analysis and Design with Applications , 1990 .

[51]  James Allan,et al.  Text classification and named entities for new event detection , 2004, SIGIR '04.

[52]  Martin Franz,et al.  Unsupervised and supervised clustering for topic tracking , 2001, SIGIR '01.

[53]  Kuo Zhang,et al.  New event detection based on indexing-tree and named entity , 2007, SIGIR.

[54]  Helena Ahonen-Myka,et al.  Applying Semantic Classes in Event Detection and Tracking , 2002 .

[55]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[56]  Yiming Yang,et al.  Learning approaches for detecting and tracking news events , 1999, IEEE Intell. Syst..

[57]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[58]  Peter Ingwersen,et al.  The development of a method for the evaluation of interactive information retrieval systems , 1997, J. Documentation.

[59]  W. Bruce Croft,et al.  Sentence level information patterns for novelty detection , 2006 .

[60]  James Allan,et al.  Improving realism of topic tracking evaluation , 2002, SIGIR '02.

[61]  Peter Willett,et al.  Readings in information retrieval , 1997 .

[62]  James Allan,et al.  Detections , Bounds , and Timelines : UMass and TDT-3 , 2000 .

[63]  Helena Ahonen-Myka,et al.  Simple Semantics in Topic Detection and Tracking , 2004, Information Retrieval.

[64]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[65]  J. Allan,et al.  On-Line New Event Detection using Single Pass Clustering , 1998 .

[66]  James Allan,et al.  Taking Topic Detection From Evaluation to Practice , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[67]  James Allan,et al.  Topic detection and tracking: event-based information organization , 2002 .

[68]  James Allan,et al.  Explorations within topic tracking and detection , 2002 .