Emotion Recognition for Japanese Short Sentences Including Slangs Based on Bag of Concepts Feature Trained by Large Web Text

The growth of Internet communication sites such as weblogs and social networking sites brought younger people especially in teens and in their 20s to create new words and to use them very often. We prepared an emotion corpus by collecting weblog article texts including new words, analyzed the corpus statistically, and proposed a method to estimate emotions of the texts. Most slang words such as Youth Slang are too ambiguous in sense classification to be registered into the existing dictionaries such as thesaurus. To cope with these words, we created a large scale of Twitter corpus and calculated sense similarities between words. We proposed to convert unknown word to semantic class id so that we might be able to process the words that were not included in the learning data. For calculation similarities between words and converting the word into word cluster id, we used the word embedding algorithms such as word2vec, or GloVe. We defined this method as a method using Bag of Concepts as feature. As a result of the evaluation experiment using several classifiers, the proposed method was proved its robustness for unknown expressions.