A Saudi Dialect Twitter Corpus for Sentiment and Emotion Analysis

In this paper, we introduce the Saudi Dialects Twitter Corpus (SDTC), comprising 5,400 tweets of Saudi dialects and Modern Standard Arabic classified for both sentiment analysis and emotion analysis. Three raters were engaged in the classification process, where they labeled each tweet according to its polarity (positive, negative, neutral, objective, spam, and not sure) and the emotion it carries using Ekman basic emotions (anger, fear, disgust, sadness, happiness, surprise, no emotion, and not sure). The data show comparable kappa and Fleiss’ kappa values for both polarity and emotion classification. The average agreement among any two raters was 65%, the average kappa for any two raters was 0.55, and Fleiss’ kappa for the three raters was 0.55. These values for kappa and Fleiss’ kappa indicate a moderate agreement. The values of kappa and Fleiss’ kappa statistics and the mapping between polarity and emotion classification in the SDTC confirm the consistency and regularity of the classification process. To the best of our knowledge, the SDTC is the first Twitter corpus for Saudi dialect labeled by three raters and classified based on the polarity of the tweets and the emotions they carry.