Detecting influenza states based on hybrid model with personal emotional factors from social networks

In this paper, we exhibit how social media data can be used to detect and analyze real-word phenomena with several data mining techniques. We investigate the real-time flu detection problem and propose a flu state detection model with personal emotional factors and semantic information (Em-Flu model). First, we extract flu-related microblog posts automatically in real-time using a hybrid model composed by Support Vector Machine with features extracted from Restricted Boltzmann Machine. In order to overcome the limitation of 140 words for posts, expect for sentiment related features, association semantic rules are also adopted as additional features, such as bag of words, negative words, degree adverbs and sentiment words dictionary. For flu state detection at specific location, we propose an unsupervised model based on personal emotional factors to figure out what state of flu in specific place. For comparison, a supervised model is also built by adopting Conditional Random Fields to decide whether a poster has "really" catch flu and what influenza stage the poster is in. Some statistic methods and prior rules are adopted in supervised model to get the flu state of specific locations by counting the number of microblog posts in different flu states. By considering personal emotional factors, spatial features and temporal patterns of influenza, the performance of unsupervised and supervised models are both improved. The system could tell when and where influenza epidemic is more likely to occur at certain time in specific locations. In different experiments results, the hybrid models show robustness and effectiveness than state-of-the-art unsupervised and supervised model only considering the number of posts.