Instant message classification in Finnish cyber security themed free-form discussion

Instant messaging enables rapid collaboration between professionals during cyber security incidents. However, monitoring discussion manually becomes challenging as the number of communication channels increases. Failure to identify relevant information from the free-form instant messages may lead to reduced situational awareness. In this paper, the problem was approached by developing a framework for classification of instant message topics of cyber security-themed discussion in Finnish. The program utilizes open source software components in morphological analysis, and subsequently converts the messages into Bag-of-Words representations before classifying them into predetermined incident categories. We compared support vector machines (SVM), multinomial naïve Bayes, and complement naïve Bayes (CNB) classification methods with five-fold cross-validation. A combination of SVM and CNB achieved classification accuracy of over 85 %, while multiclass SVM achieved 87 % accuracy. The implemented program recognizes cyber security-related messages in IRC chat rooms and categorizes them accordingly.