Near real-time detection of crisis situations

When disaster strikes, be it natural or man-made, the immediacy of notifying emergency professionals is critical to be able to best initiate a helping response. As has social media become ubiquitous in the recent years, so have affected citizens become fast reporters of an incident. However, wanting to exploit such `citizen sensors' for identifying a crisis situation comes at a price of having to sort, in near real-time, through vast amounts of mostly unrelated, and highly unstructured information exchanged among individuals around the world. Identifying bursts in conversations can, however, decrease the burden by pinpointing an event of potential interest. Still, the vastness of information keeps the computational requirements for such procedures, even if optimized, too high for a non-distributed approach. This is where currently emerging, real-time focused distributed processing systems may excel. This paper elaborates on the possible practices, caveats, and recommendations for engineering a cloud-centric application on one such system. We used the distributed real-time computation system Apache Storm and its Trident API in conjunction with detecting crisis situations by identifying bursts in a streamed Twitter communication. We contribute a system architecture for the suggested application, and a high level description of its components' implementation.

[1]  Bu-Sung Lee,et al.  Event Detection in Twitter , 2011, ICWSM.

[2]  Jie Yin,et al.  Using Social Media to Enhance Emergency Situation Awareness , 2012, IEEE Intelligent Systems.

[3]  Wael Khreich,et al.  A Survey of Techniques for Event Detection in Twitter , 2015, Comput. Intell..

[4]  Gottfried Vossen,et al.  Web Information Systems Engineering - WISE 2009, 10th International Conference, Poznan, Poland, October 5-7, 2009. Proceedings , 2009, WISE.

[5]  Mitsuru Ishizuka,et al.  Keyword extraction from a single document using word co-occurrence statistical information , 2004, Int. J. Artif. Intell. Tools.

[6]  Charu C. Aggarwal,et al.  Event Detection in Social Streams , 2012, SDM.

[7]  Ke Wang,et al.  TopicSketch: Real-Time Bursty Topic Detection from Twitter , 2013, 2013 IEEE 13th International Conference on Data Mining.

[8]  Christopher Brewster,et al.  Ontologies for crisis management: A review of state of the art in ontology design and usability , 2013, ISCRAM.

[9]  KONG XIANGSHENG MICROBLOG MINING BASED ON CLOUD COMPUTING TECHNOLOGIES : MESOS AND HADOOP , 2013 .

[10]  Vanessa Evers,et al.  Canary in a coal mine: monitoring air quality and detecting environmental incidents by harvesting Twitter , 2011, CHI Extended Abstracts.

[11]  Werner Retschitzegger,et al.  CrowdSA — towards adaptive and situation-driven crowd-sensing for disaster situation awareness , 2015, 2015 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision.

[12]  Y. Matsuo,et al.  Tweet trend analysis in an emergency situation , 2011, SWID '11.

[13]  Werner Retschitzegger,et al.  Crowd-Sensing Meets Situation Awareness: A Research Roadmap for Crisis Management , 2015, 2015 48th Hawaii International Conference on System Sciences.

[14]  Yang Song,et al.  Topical Keyphrase Extraction from Twitter , 2011, ACL.

[15]  Vassilis Kostakos,et al.  CrisisTracker: Crowdsourced social media curation for disaster awareness , 2013, IBM J. Res. Dev..

[16]  Ee-Peng Lim,et al.  Analyzing feature trajectories for event detection , 2007, SIGIR.

[17]  Miles Osborne,et al.  Streaming First Story Detection with application to Twitter , 2010, NAACL.

[18]  Judy Qiu,et al.  Parallel Clustering of High-Dimensional Social Media Data Streams , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[19]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[20]  D. D. Wrachien,et al.  Ontology For Flood Management: A Proposal , 2012 .

[21]  Pádraig Cunningham,et al.  High-Throughput , Web-Scale Data Stream Clustering , 2014 .

[22]  Hila Becker,et al.  Learning similarity metrics for event identification in social media , 2010, WSDM '10.

[23]  Werner Retschitzegger,et al.  Mining the disaster hotspots - situation-adaptive crowd knowledge extraction for crisis management , 2016, 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA).

[24]  Ee-Peng Lim,et al.  Finding Bursty Topics from Microblogs , 2012, ACL.