“Making the News”: Identifying Noteworthy Events in News Articles

Most events described in a news article are background events ‐ only a small number are noteworthy, and a even smaller number serve as the trigger for writing of that article. Although these events are difficult to identify, they are crucial to NLP tasks such as first story detection, document summarization and event coreference, and to many applications of event analysis that depend on event counting and identifying trends. In this work, we introduce the notion of news-peg, a concept borrowed from the political science literature, in an attempt to remedy this problem. A news-peg is an event which prompted the author to write the article, and it serves as a more fine-grained measure of noteworthiness than a summary. We describe a new task of news-peg identification and release an annotated dataset for its evaluation. We formalize an operational definition of a news-peg, on which human annotators achieve high inter-annotator agreement (over 80%), and present a rule-based system for this task, which exploits syntactic features derived from established journalistic devices.