Automated classification of social network messages into Smart Cities dimensions

Abstract A Smart City can be defined as a high-tech city with several public and private services capable to strategically solve (or mitigate) problems normally generated by rapid urbanization. Different models of indicators have been developed to follow cities’ evolution to become a Smart City. An example of such model is the standard 37120 from the International Organization for Standardization (ISO) that proposes a set of dimensions and indicators (e.g. Transportation, Recreation, Solid Waste) for services and quality of life for sustainable cities and communities. It has been common to find official social network profiles of organizations and governmental entities related to the services they provide or are responsible for (water, waste, transportation, cultural events, etc.) and that are used by citizens as a gateway to directly interact and communicate their complains and problems about those services. The present paper proposes to apply machine learning algorithms over the urban data generated by social networks in order to create classifiers to automatically categorize citizens messages according to the different cities services dimensions. For that, two distinct text datasets in Portuguese were collected from two social networks: Twitter (1,950 tweets) and Colab.re (65,066 posts). The texts were mapped according to the different ISO 37120 categories, preprocessed and mined through the use of 8 algorithms implemented in Scikit-Learn. Initial results pointed out the feasibility of the proposal with models achieving average F1-measures around 55% for F1-macro and 78% for F1-micro when using Linear Vector Classification, Logistic Regression, Decision Tree and Complement Naive Bayes. However, as the datasets were highly unbalanced, the performances of the models vary significantly for each ISO category, with the best results occurring for Wastewater, Water & Sanitation, Energy and Transportation. The classifiers generated here can be integrated on a number of different city services and systems such as: governmental support decision systems, customer complain systems, communities dashboards, police offices, transportation’s companies, cultural producers, environmental agencies, and recyclers’ companies.