Twitter: A New Online Source of Automatically Tagged Data for Conversational Speech Emotion Recognition

In the space of affect detection in multimedia, there is a strong demand for more tagged data in order to better understand human emotions, the way they are expressed, and approaches for detecting them automatically. Unfortunately, emotion datasets are typically small due to the manual process of annotating them with emotional labels. In response, we present for the first time the application of automatically tagged Twitter data to the problem of speech emotion recognition (SER). SER has been shown to benefit from the combination of acoustic and linguistic features, albeit when the linguistic training data is from the same database as the test data. Using the presence of emoticons for automatic tagging, we compile a corpus of over 800,000 tweets that is totally independent from our evaluation database. By supplementing an acoustic classifier with linguistic information, we classify the spontaneous content within the USC-IEMOCAP corpus on valence and activation descriptors. With comparison to prior literature, we demonstrate performance improvements for valence of 2% and 6% over an acoustic-only system, using linguistic training data from Twitter and IEMOCAP respectively.