Corpus of News Articles Annotated with Article Level Sentiment

Research on sentiment analysis is in its mature status. Studies on this topic have proposed various solutions and datasets to guide machine-learning approaches. However, so far the sentiment scoring is restricted to the level of short textual units such as sentences. Our comparison shows that there is a huge gap between machines and human judges when the task is to determine sentiment scores of a longer text such as a news article. To close this gap, we propose a new human-annotated dataset containing 250 news articles with sentiment labels at article level. Each article is annotated by at least 10 people. The articles are evenly divided into fake and non-fake categories. Our investigation on this corpus shows that fake articles are significantly more sentimental than non-fake ones. The dataset will be made publicly available. Copyright © 2019 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors. In: A. Aker, D. Albakour, A. Barrón-Cedeño, S. Dori-Hacohen, M. Martinez, J. Stray, S. Tippmann (eds.): Proceedings of the NewsIR’19 Workshop at SIGIR, Paris, France, 25-July-2019, published at http://ceur-ws.org