Burst prediction from Weibo: A crowd-sensing and tweet-centric method

Online social media such as Weibo generates copious and to-the-minute information about real-world events of all kinds. How to effectively and efficiently detect emergent events from massive tweet streams is now drawing attention from various sources of every interest. Despite the wealth of previous research work, bursty topic detection (or burst prediction) remains a huge challenge because of certain qualities in the data such as sparseness of useable information and an enormous amount of noise in each data set. In particular, the conventional term-frequency-based approaches may not be appropriate in this context, since the propagation of Weibo events is typically driven by a few influential posts, which are often buried among irrelevant noisy tweets with frequent but trivial terms. Other commonly used methods, such as topic models, may also fail to detect bursty events due to the high sparseness of tweets and high computational complexity. In light of this, this paper proposes a crowd-sensing, tweet-centric method for burst prediction. That is, we first select influential users on Weibo as social sensors to perceive bursty events via their posts or reposts. This is indeed crucial for excluding the interference of unimportant tweets and reducing computational costs. All the tweets are then subject to the filtering of uninteresting topics and the remainder is then monitored for possible bursts according to the modeling of the time-series of retweeted occurrences. Extensive experiments on real-life huge Weibo datasets show both the efficiency and effectiveness of our approach. In particular, the tweet-centric philosophy of our method provides rich semantics to the detected bursts and thus is of great value in practice.