Event Detection in Twitter

Twitter, as a form of social media, is fast emerging in recent years. Users are using Twitter to report real-life events. This paper focuses on  detecting those events by analyzing the text stream in Twitter. Although event detection has long been a research topic, the characteristics of Twitter make it a non-trivial task. Tweets reporting such events are usually overwhelmed by high flood of meaningless “babbles”. Moreover, event detection algorithm needs to be scalable given the sheer amount of tweets. This paper attempts to tackle these challenges with EDCoW (Event Detection with Clustering of Wavelet-based Signals). EDCoW builds signals for individual words by applying wavelet analysis on the frequencybased raw signals of the words. It then filters away the trivial words by looking at their corresponding signal autocorrelations. The remaining words are then clustered to form events with a modularity-based graph partitioning technique. Experimental results show promising result of EDCoW.

[1]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[2]  Hector Garcia-Molina,et al.  Overview of multidatabase transaction management , 2005, The VLDB Journal.

[3]  M E J Newman,et al.  Fast algorithm for detecting community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Ilse C. F. Ipsen,et al.  Mathematical properties and analysis of Google's PageRank , 2008 .

[5]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[6]  Ling Chen,et al.  Event detection from flickr data through wavelet-based spatial analysis , 2009, CIKM.

[7]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[8]  E. Basar,et al.  Wavelet entropy: a new tool for analysis of short duration brain electrical signals , 2001, Journal of Neuroscience Methods.

[9]  Jon M. Kleinberg,et al.  Bursty and Hierarchical Structure in Streams , 2002, Data Mining and Knowledge Discovery.

[10]  Andreas M. Kaplan,et al.  The early bird catches the news: Nine things you should know about micro-blogging , 2011 .

[11]  W. Kilmer.  A Friendly Guide To Wavelets , 1998, Proceedings of the IEEE.

[12]  H. M. Walker,et al.  Studies in the history of statistical method , 1930 .

[13]  Philip S. Yu,et al.  Parameter Free Bursty Events Detection in Text Streams , 2005, VLDB.

[14]  Ee-Peng Lim,et al.  Analyzing feature trajectories for event detection , 2007, SIGIR.

[15]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[17]  Yiming Yang,et al.  A study of retrospective and on-line event detection , 1998, SIGIR '98.

[18]  Miles Osborne,et al.  Streaming First Story Detection with application to Twitter , 2010, NAACL.

[19]  Lei Zhang,et al.  LivePulse: tapping social media for sentiments in real-time , 2011, WWW.

[20]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[21]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.