Flat and Hierarchical Classifiers for Detecting Emotion in Tweets

Social media are more and more frequently used by people to express their feelings in the form of short messages. This has raised interest in emotion detection, with a wide range of applications among which the assessment of users' moods in a community is perhaps the most relevant. This paper proposes a comparison between two approaches to emotion classification in tweets, taking into account six basic emotions. Additionally, it proposes a completely automated way of creating a reliable training set, usually a tedious task performed manually by humans. In this work, training datasets have been first collected from the web and then automatically filtered to exclude ambiguous cases, using an iterative procedure. Test datasets have been similarly collected from the web, but annotated manually. Two approaches have then been compared. The first is based on a direct application of a single "flat" seven-output classifier, which directly assigns one of the emotions to the input tweet, or classifies it as "objective", when it appears not to express any emotion. The other approach is based on a three-level hierarchy of four specialized classifiers, which reflect a priori relationships between the target emotions. In the first level, a binary classifier isolates subjective expressing emotions from objective tweets. In the second, another binary classifier labels subjective tweets as positive or negative. Finally, in the third, one ternary classifier labels positive tweets as expressing joy, love, or surprise, while another classifies negative tweets as expressing anger, fear, or sadness. Our tests show that the a priori domain knowledge embedded into the hierarchical classifier makes it significantly more accurate than the flat classifier.