Evaluation results for a Social Media Analyst Responding Tool

We take a human-centered design approach to develop a fully functional prototype, SMART (“Social Media Analyst Responding Tool”), informed by emergency practitioners. The prototype incorporates machine learning techniques to identify relevant information during emergencies. In this paper, we report the result of a user study to gather qualitative feedback on SMART. The evaluation results offer recommendations into the design of social media analysis tools for emergencies. The evaluation findings show the interest of emergency practitioners into designing such solutions; it reflects their need to not only identify relevant information but also to further perceive the outcome of their actions in social media. We found out there is a notable emphasis on the sentiment from these practitioners and social media analysis tools need to do a better job of handling negative sentiment within the emergency concept.

[1]  Marina Kogan,et al.  Think Local, Retweet Global: Retweeting by the Geographically-Vulnerable during Hurricane Sandy , 2015, CSCW.

[2]  Vassilis Kostakos,et al.  CrisisTracker: Crowdsourced social media curation for disaster awareness , 2013, IBM J. Res. Dev..

[3]  Laura Hokkanen,et al.  Social media in crisis management - The iSAR+ project survey , 2014, ISCRAM.

[4]  Barbara Poblete,et al.  Twitter under crisis: can we trust what we RT? , 2010, SOMA '10.

[5]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[6]  Yikun Liu,et al.  Top health trends: An information visualization tool for awareness of local health trends , 2013, ISCRAM.

[7]  Larry L. Constantine,et al.  Software for Use - A Practical Guide to the Models and Methods of Usage-Centered Design , 1999 .

[8]  Axel Schulz,et al.  I See a Car Crash: Real-Time Detection of Small Scale Incidents in Microblogs , 2013, ESWC.

[9]  Geert-Jan Houben,et al.  Semantics + filtering + search = twitcident. exploring information in social web streams , 2012, HT '12.

[10]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[11]  Frank Maurer,et al.  Social Media Use During Emergency Response - Insights from Emergency Professionals , 2016, I3E.

[12]  Starr Roxanne Hiltz,et al.  Red Tape: Attitudes and Issues Related to Use of Social Media by U.S. County-Level Emergency Managers , 2015, ISCRAM.

[13]  Leysia Palen,et al.  Natural Language Processing to the Rescue? Extracting "Situational Awareness" Tweets During Mass Emergency , 2011, ICWSM.

[14]  Clayton Lewis,et al.  TASK-CENTERED USER INTERFACE DESIGN A Practical Introduction , 2006 .

[15]  Muhammad Imran,et al.  Coordinating human and machine intelligence to classify microblog communications in crises , 2014, ISCRAM.

[16]  Charu C. Aggarwal,et al.  Mining Text Data , 2012, Springer US.

[17]  Apoorve Chokshi.  Designing Social Media Tools for Emergency Response , 2015 .

[18]  Volker Wulf,et al.  New Perspectives in End-User Development , 2017, Springer International Publishing.

[19]  Xiao Zhang,et al.  SensePlace2: GeoTwitter analytics support for situational awareness , 2011, 2011 IEEE Conference on Visual Analytics Science and Technology (VAST).

[20]  SaltonGerard,et al.  Term-weighting approaches in automatic text retrieval , 1988 .

[21]  David Ratcliffe,et al.  Finding Fires with Twitter , 2013, ALTA.

[22]  Sarah Vieweg,et al.  Processing Social Media Messages in Mass Emergency , 2014, ACM Comput. Surv..

[23]  Jie Yin,et al.  ESA: emergency situation awareness via microbloggers , 2012, CIKM.

[24]  C. Brodsky.  The Discovery of Grounded Theory: Strategies for Qualitative Research , 1968 .

[25]  Christian Reuter,et al.  Semi-automatic alerts and notifications for emergency services based on cross-platform social media data - evaluation of a prototype , 2016, GI-Jahrestagung.

[26]  Rohan Shah,et al.  Designing an Application for Social Media Needs in Emergency Public Information Work , 2016, GROUP.

[27]  Jason Thornton,et al.  Feedback-based social media filtering tool for improved situational awareness , 2016, 2016 IEEE Symposium on Technologies for Homeland Security (HST).

[28]  Thomas Ludwig,et al.  Emergency services' attitudes towards social media: A quantitative and qualitative survey across Europe , 2016, Int. J. Hum. Comput. Stud..

[29]  Jie Yin,et al.  Emergency situation awareness from twitter for crisis management , 2012, WWW.

[30]  Thomas Ludwig,et al.  End-User Development and Social Big Data - Towards Tailorable Situation Assessment with Social Media , 2017, New Perspectives in End-User Development.

[31]  Starr Roxanne Hiltz,et al.  Dealing with information overload when using social media for emergency management: Emerging solutions , 2013, ISCRAM.

[32]  Leysia Palen,et al.  Twitter adoption and use in mass convergence and emergency events , 2009 .

[33]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[34]  Christian Reuter,et al.  Retrospective Review and Future Directions for Crisis Informatics , 2021, Information Refinement Technologies for Crisis Informatics.

[35]  Stefan Stieglitz,et al.  Sense‐Making in Social Media During Extreme Events , 2018 .

[36]  John M. Carroll,et al.  Making Use: Scenario-Based Design of Human-Computer Interactions , 2000 .