Face off: Travel Habits, Road Conditions and Traffic City Characteristics Bared Using Twitter

The adequacy of traditional transport related issues detection is often limited by physical sparse sensor coverage and reporting incident/issues to the emergency response system is labor intensive. The social media tweet text have been mined so as to identify the complaints regarding various road transportation issues of traffic, accident, and potholes. In order to identify and segregate tweets related to different issues, keyword-based approaches have been used previously, but these methods are solely dependent on seed keywords which are manually given and these set of keywords are not sufficient to cover all tweets posts. So, to overcome this issue, a novel approach has been proposed that captures the semantic context through dense word embedding by employing word2vec model. However, the process of tweet segregation on the basis of semantic similar keywords may suffer from the problem of pragmatic ambiguity. To handle this, Word2Vec model has been applied to match the semantically similar tweets with respect to each category. Furthermore, the hotspots have been identified corresponding to each category. However, due to the scarcity of geo-tagged tweets, we have proposed a hybrid method which amalgamates Named Entity Recognition (NER), Part of speech (POS), and Regular Expression (RE) to extract the location information from the tweet textual content. Due to the lack of availability of the ground truth dataset, model feasibility has been validated from the existing data records (i.e., published by government official accounts and reported on news media) and the evaluation results signify that the stated approach identifies few additional hotspots as compared to the existing reports while analyzing the tweets.

[1]  Axel Schulz,et al.  I See a Car Crash: Real-Time Detection of Small Scale Incidents in Microblogs , 2013, ESWC.

[2]  Chenliang Li,et al.  Fine-grained location extraction from tweets with temporal awareness , 2014, SIGIR.

[3]  Xiaohui Yan,et al.  A biterm topic model for short texts , 2013, WWW.

[4]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[5]  Ming Zhou,et al.  Named entity recognition for tweets , 2013, TIST.

[6]  Judith Gelernter,et al.  Geocoding location expressions in Twitter messages: A preference learning method , 2014, J. Spatial Inf. Sci..

[7]  Anders Karlström,et al.  A new information theoretical measure of global and local spatial association , 2000 .

[8]  Catherine A. Calder,et al.  Beyond Moran's I: Testing for Spatial Dependence Based on the Spatial Autoregressive Model , 2007 .

[9]  Bu-Sung Lee,et al.  Event Detection in Twitter , 2011, ICWSM.

[10]  Rohan Kumar,et al.  Tweeting Traffic: Analyzing Twitter for generating real-time city traffic insights and predictions , 2015, CODS Companion Volume.

[11]  Geert-Jan Houben,et al.  Twitcident: fighting fire with information from social web streams , 2012, WWW.

[12]  Christian Rohrdantz,et al.  Getting there first : real-time detection of real-world incidents on Twitter , 2012 .

[13]  Feng Chen,et al.  From Twitter to detector: real-time traffic incident detection using social media data , 2016 .

[14]  Dan Roth,et al.  Design Challenges and Misconceptions in Named Entity Recognition , 2009, CoNLL.

[15]  Jie Yin,et al.  Location extraction from disaster-related microblogs , 2013, WWW.

[16]  Dawei Wang,et al.  Crime hotspot mapping using the crime related factors—a spatial data mining approach , 2012, Applied Intelligence.

[17]  Ankush Mittal,et al.  Construction of a Semi-Automated model for FAQ Retrieval via Short Message Service , 2015, FIRE.

[18]  Ming Zhou,et al.  Recognizing Named Entities in Tweets , 2011, ACL.

[19]  Judith Gelernter,et al.  Cross-lingual geo-parsing for non-structured data , 2013, GIR '13.

[20]  Eleonora D'Andrea,et al.  Real-Time Detection of Traffic From Twitter Stream Analysis , 2015, IEEE Transactions on Intelligent Transportation Systems.

[21]  Hua Wang,et al.  Enhancing Traffic Incident Detection by Using Spatial Point Pattern Analysis on Social Media , 2015 .

[22]  Jing Gao,et al.  A deep learning approach for detecting traffic accidents from social media data , 2018, ArXiv.

[23]  Oren Etzioni,et al.  Named Entity Recognition in Tweets: An Experimental Study , 2011, EMNLP.

[24]  Robert L. Mercer,et al.  Class-Based n-gram Models of Natural Language , 1992, CL.

[25]  Zuo Zhang,et al.  Extraction of traffic information from social media interactions: Methods and experiments , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[26]  Abdullah Kurkcu,et al.  Extended Implementation Method for Virtual Sensors: Web-Based Real-Time Transportation Data Collection and Analysis for Incident Management , 2015 .

[27]  Xiao Wang,et al.  A convolutional neural network for traffic information sensing from social media text , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[28]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[29]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[30]  Shervin Malmasi,et al.  Location Mention Detection in Tweets and Microblogs , 2015, PACLING.

[31]  Scott Grosenick,et al.  Real-Time Traffic Prediction Improvement through Semantic Mining of Social Networks , 2012 .

[32]  Shourya Roy,et al.  A survey of types of text noise and techniques to handle noisy text , 2009, AND '09.

[33]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[34]  Ming Ni,et al.  Using Social Media to Predict Traffic Flow under Special Event Conditions , 2013 .

[35]  Di Wang,et al.  Real-Time Traffic Event Detection From Social Media , 2017, ACM Trans. Internet Techn..

[36]  Michael Gertz,et al.  EvenTweet: Online Localized Event Detection from Twitter , 2013, Proc. VLDB Endow..

[37]  Wei Shen,et al.  Improving Traffic Prediction with Tweet Semantics , 2013, IJCAI.

[38]  Ricardo Jardim-Goncalves,et al.  Twitter mining for traffic events detection , 2015, 2015 Science and Information Conference (SAI).

[39]  Judith Gelernter,et al.  An algorithm for local geoparsing of microtext , 2013, GeoInformatica.