Social media research: The application of supervised machine learning in organizational communication research

Despite the online availability of data, analysis of this information in academic research is arduous. This article explores the application of supervised machine learning (SML) to overcome challenges associated with online data analysis. In SML classifiers are used to categorize and code binary data. Based on a case study of Dutch employees' work-related tweets, this paper compares the coding performance of three classifiers, Linear Support Vector Machine, Naive Bayes, and logistic regression. The performance of these classifiers is assessed by examining accuracy, precision, recall, the area under the precision-recall curve, and Krippendorf's Alpha. These indices are obtained by comparing the coding decisions of the classifier to manual coding decisions. The findings indicate that the Linear Support Vector Machine and Naive Bayes classifiers outperform the logistic regression classifier. This study also compared the performance of these classifiers based on stratified random samples and random samples of training data. The findings indicate that in smaller training sets stratified random training samples perform better than random training samples, in large training sets (nź=ź4000) random samples yield better results. Finally, the Linear Support Vector Machine classifier was trained with 4000 tweets and subsequently used to categorize 578,581 tweets obtained from 430 employees. Supervised Machine Learning (SML) is suitable for coding social media content.Linear Support Vector Machine and Naive Bayes classifiers can be trained using 4000 training tweets.SML enables researchers to escalate the scope of their research without compromising data size or depth.Linear Support Vector Machine and Naive Bayes outperform the logistic regression classifier.Classifiers perform better based on stratified random samples compared to random samples when training samples are small.