Analyzing Social Media Texts and Images to Assess the Impact of Flash Floods in Cities

Computer Vision and Image Processing are emerging research paradigms. The increasing popularity of social media, micro- blogging services and ubiquitous availability of high-resolution smartphone cameras with pervasive connectivity are propelling our digital footprints and cyber activities. Such online human footprints related with an event-of-interest, if mined appropriately, can provide meaningful information to analyze the current course and pre- and post- impact leading to the organizational planning of various real-time smart city applications. In this paper, we investigate the narrative (texts) and visual (images) components of Twitter feeds to improve the results of queries by exploiting the deep contexts of each data modality. We employ Latent Semantic Analysis (LSA)-based techniques to analyze the texts and Discrete Cosine Transformation (DCT) to analyze the images which help establish the cross-correlations between the textual and image dimensions of a query. While each of the data dimensions helps improve the results of a specific query on its own, the contributions from the dual modalities can potentially provide insights that are greater than what can be obtained from the individual modalities. We validate our proposed approach using real Twitter feeds from a recent devastating flash flood in Ellicott City near the University of Maryland campus. Our results show that the images and texts can be classified with 67\% and 94\% accuracies respectively

[1]  Charu C. Aggarwal,et al.  Unveiling polarization in social networks: A matrix factorization approach , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[2]  Daniel A. Keim,et al.  State-of-the-Art Report of Visual Analysis for Event Detection in Text Data Streams , 2014, EuroVis.

[3]  Cécile Favre,et al.  Event detection, tracking, and visualization in Twitter: a mention-anomaly-based approach , 2015, Social Network Analysis and Mining.

[4]  Maximilian Walther,et al.  Geo-spatial Event Detection in the Twitter Stream , 2013, ECIR.

[5]  Michael S. Bernstein,et al.  Processing and visualizing the data in tweets , 2011, SGMD.

[6]  Hermann Ney,et al.  Automatic categorization of medical images for content-based retrieval and data mining. , 2005, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[7]  Archan Misra,et al.  Can Instagram posts help characterize urban micro-events? , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[8]  Geert-Jan Houben,et al.  Twitcident: fighting fire with information from social web streams , 2012, WWW.

[9]  Ross Maciejewski,et al.  Visualizing Social Media Sentiment in Disaster Scenarios , 2015, WWW.

[10]  Jie Yin,et al.  Location extraction from disaster-related microblogs , 2013, WWW.

[11]  Ross Maciejewski,et al.  Understanding Twitter data with TweetXplorer , 2013, KDD.

[12]  Mohammad Ali Abbasi,et al.  TweetTracker: An Analysis Tool for Humanitarian and Disaster Relief , 2011, ICWSM.

[13]  Shiguang Wang,et al.  Social Fusion: Integrating Twitter and Instagram for Event Monitoring , 2017, 2017 IEEE International Conference on Autonomic Computing (ICAC).

[14]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.