On-line New Event Detection, Clustering, and Tracking TITLE2:

In this work, we discuss and evaluate solutions to text classification problems associated with the events that are reported in on-line sources of news. We present solutions to three related classification problems: new event detection, event clustering, and event tracking. The primary focus of this thesis is new event detection, where the goal is to identify news stories that have not previously been reported, in a stream of broadcast news comprising radio, television, and newswire. We present an algorithm for new event detection, and analyze the effects of incorporating domain properties into the classification algorithm. We explore a solution that models the temporal relationship between news stories, and investigate the use of proper noun phrase extraction to capture the `who'', `what'', `when'', and `where'' contained in news. Our results for new event detection suggest that previous approaches to document clustering provide a good basis for an approach to new event detection, and that further improvements to classification accuracy are obtained when the domain properties of broadcast news are modeled. New event detection is related to the problem of event clustering, where the goal is to group stories that discuss the same event. We investigate on-line clustering as an approach to new event detection, and re-evaluate existing cluster comparison strategies previously used for document retrieval. Our results suggest that these strategies produce different groupings of events, and that the on-line single-link strategy extended with a model for domain properties is faster and more effective than other approaches. In this dissertation, we explore several text representation issues in the context of event tracking, where a classifier for an event is formulated from one or more sample stories. The classifier is used to monitor the subsequent news stream for documents related to the event. We discuss different approaches to classifier formulation, and compare feature selection and weight-learning steps as extensions to a baseline process used for new event detection. In addition, we evaluate an unsupervised adaptive approach to event tracking that captures the property of event evolution in broadcast news. We introduce a theoretical framework for automatic threshold estimation for our tracking algorithm, viewing the threshold as a statistic of the incoming data stream. Bias defined in terms of threshold estimators can be identified with statistical and learning techniques using our representation for text. Two methods are presented that learn estimator bias and result in improved classification accuracy for tracking. The implementations of our approaches to on-line new event detection, clustering, and tracking have been evaluated in comparison to other systems, and we present cross-system comparisons for all three classification problems. In general, the results using our approaches compared favorably to other approaches for each problem.