Real-Time Traffic Event Detection From Social Media

Smart communities are composed of groups, organizations, and individuals who share information and make use of that shared information for better decision making. Shared information can come from many sources, particularly, but not exclusively, from sensors and social media. Social media has become an important source of near-instantaneous user-generated information that can be shared and analyzed to support better decision making. One domain where social media data can add value is transportation and traffic management. This article looks at the exploitation of Twitter data in the traffic reporting domain. A key challenge is how to identify relevant information from a huge amount of user-generated data and then analyze the relevant data for automatic geocoded incident detection. The article proposes an instant traffic alert and warning system based on a novel latent Dirichlet allocation (LDA) approach (“tweet-LDA”). The system is evaluated and shown to perform better than related approaches.

[1]  Fan Yu,et al.  Towards large-scale twitter mining for drug-related adverse events , 2012, SHB '12.

[2]  Yutaka Matsuo,et al.  Tweet Analysis for Real-Time Event Detection and Earthquake Reporting System Development , 2013, IEEE Transactions on Knowledge and Data Engineering.

[3]  Thomas L. Griffiths,et al.  Online Inference of Topics with Latent Dirichlet Allocation , 2009, AISTATS.

[4]  John Davies,et al.  Event identification and assertion from social media using auto-extendable knowledge base , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[5]  Timothy Baldwin,et al.  Lexical Normalisation of Short Text Messages: Makn Sens a #twitter , 2011, ACL.

[6]  Hiroshi Nakagawa,et al.  Deterministic Single-Pass Algorithm for LDA , 2010, NIPS.

[7]  Brian D. Davison,et al.  Empirical study of topic modeling in Twitter , 2010, SOMA '10.

[8]  Kirill Kireyev.  Applications of Topics Models to Analysis of Disaster-Related Twitter Data , 2009 .

[9]  Danushka Bollegala,et al.  Multi-tweet Summarization of Real-Time Events , 2013, 2013 International Conference on Social Computing.

[10]  Hakan Ferhatosmanoglu,et al.  Short text classification in twitter to improve information filtering , 2010, SIGIR.

[11]  Di Wang,et al.  Semi-Supervised Latent Dirichlet Allocation and Its Application for Document Classification , 2012, 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[12]  Oren Etzioni,et al.  Named Entity Recognition in Tweets: An Experimental Study , 2011, EMNLP.

[13]  Efthimis N. Efthimiadis,et al.  Conversational tagging in twitter , 2010, HT '10.

[14]  David M. Blei,et al.  Supervised Topic Models , 2007, NIPS.

[15]  Hongfei Yan,et al.  Comparing Twitter and Traditional Media Using Topic Models , 2011, ECIR.

[16]  Carolyn Penstein Rosé,et al.  Detecting offensive tweets via topical feature discovery over a large scale twitter corpus , 2012, CIKM.

[17]  Thomas L. Griffiths,et al.  Probabilistic author-topic models for information discovery , 2004, KDD.

[18]  Y. Matsuo,et al.  Real-time event extraction for driving information from social sensors , 2012, 2012 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER).

[19]  Francis R. Bach,et al.  Online Learning for Latent Dirichlet Allocation , 2010, NIPS.

[20]  Takumi Ichimura,et al.  A generation method of filtering rules of Twitter via smartphone based Participatory Sensing system for tourist by interactive GHSOM and C4.5 , 2018, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[21]  Brendan T. O'Connor,et al.  TweetMotif: Exploratory Search and Topic Summarization for Twitter , 2010, ICWSM.

[22]  Haixun Wang,et al.  Tracking and Connecting Topics via Incremental Hierarchical Dirichlet Processes , 2011, 2011 IEEE 11th International Conference on Data Mining.

[23]  Aron Culotta,et al.  Towards detecting influenza epidemics by analyzing Twitter messages , 2010, SOMA '10.

[24]  Feng Chen,et al.  From Twitter to detector: real-time traffic incident detection using social media data , 2016 .

[25]  Christophe G. Giraud-Carrier,et al.  Identifying Health-Related Topics on Twitter - An Exploration of Tobacco-Related Tweets as a Test Topic , 2011, SBP.

[26]  Max Kaufmann.  Syntactic Normalization of Twitter Messages , 2010 .

[27]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[28]  Jeffrey Nichols,et al.  Home Location Identification of Twitter Users , 2014, TIST.

[29]  Jie Yin,et al.  Using Social Media to Enhance Emergency Situation Awareness , 2012, IEEE Intelligent Systems.

[30]  Gisele L. Pappa,et al.  Inferring the Location of Twitter Messages Based on User Relationships , 2011, Trans. GIS.

[31]  Mark Dredze,et al.  You Are What You Tweet: Analyzing Twitter for Public Health , 2011, ICWSM.

[32]  Eleonora D'Andrea,et al.  Real-Time Detection of Traffic From Twitter Stream Analysis , 2015, IEEE Transactions on Intelligent Transportation Systems.