An unsupervised multilingual approach for online social media topic identification

An unsupervised multilingual approach to identify topics on Twitter is proposed.Localised language can be leveraged for identifying relevant and important topics.Joint term ranking coupled with DPMM clustering consistently performed well.Multilingual sentiment analysis is essential to understand sentiment on the ground.Topics coverage of social media and main stream media does not always stay the same. Social media data can be valuable in many ways. However, the vast amount of content shared and the linguistic variants of languages used on social media are making it very challenging for high-value topics to be identified. In this paper, we present an unsupervised multilingual approach for identifying highly relevant terms and topics from the mass of social media data. This approach combines term ranking, localised language analysis, unsupervised topic clustering and multilingual sentiment analysis to extract prominent topics through analysis of Twitter's tweets from a period of time. It is observed that each of the ranking methods tested has their strengths and weaknesses, and that our proposed Joint ranking method is able to take advantage of the strengths of the ranking methods. This Joint ranking method coupled with an unsupervised topic clustering model is shown to have the potential to discover topics of interest or concern to a local community. Practically, being able to do so may help decision makers to gauge the true opinions or concerns on the ground. Theoretically, the research is significant as it shows how an unsupervised online topic identification approach can be designed without much manual annotation effort, which may have great implications for future development of expert and intelligent systems.