Social Media: New Perspectives to Improve Remote Sensing for Emergency Response

Remote sensing is a powerful technology for Earth observation (EO), and it plays an essential role in many applications, including environmental monitoring, precision agriculture, resource managing, urban characterization, disaster and emergency response, etc. However, due to limitations in the spectral, spatial, and temporal resolution of EO sensors, there are many situations in which remote sensing data cannot be fully exploited, particularly in the context of emergency response (i.e., applications in which real/near-real-time response is needed). Recently, with the rapid development and availability of social media data, new opportunities have become available to complement and fill the gaps in remote sensing data for emergency response. In this paper, we provide an overview on the integration of social media and remote sensing in time-critical applications. First, we revisit the most recent advances in the integration of social media and remote sensing data. Then, we describe several practical case studies and examples addressing the use of social media data to improve remote sensing data and/or techniques for emergency response.

[1]  Vikram Krishnamurthy,et al.  A Tutorial on Interactive Sensing in Social Networks , 2014, IEEE Transactions on Computational Social Systems.

[2]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Binbin Pan,et al.  A novel discriminant criterion based on feature fusion strategy for face recognition , 2015, Neurocomputing.

[4]  Nima Tajbakhsh,et al.  Comparing two classes of end-to-end machine-learning models in lung nodule detection and classification: MTANNs vs. CNNs , 2017, Pattern Recognit..

[5]  Ioannis K. Tsanis,et al.  Developing Geographic Information Systems for Land Use Impact Assessment in Flooding Conditions , 1998 .

[6]  Xiao Li,et al.  Machine Learning Paradigms for Speech Recognition: An Overview , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[7]  Anthony C. Robinson,et al.  Leveraging geospatially-oriented social media communications in disaster response , 2012, ISCRAM.

[8]  Atsuo Yoshitaka,et al.  A Survey on Content-Based Retrieval for Multimedia Databases , 1999, IEEE Trans. Knowl. Data Eng..

[9]  Chin-Hui Lee,et al.  A unified approach to transfer learning of deep neural networks with applications to speaker adaptation in automatic speech recognition , 2016, Neurocomputing.

[10]  Vassilis Kostakos,et al.  CrisisTracker: Crowdsourced social media curation for disaster awareness , 2013, IBM J. Res. Dev..

[11]  Xizhao Wang,et al.  Learning from big data with uncertainty - editorial , 2015, J. Intell. Fuzzy Syst..

[12]  Doris Dransch,et al.  Utilising social media contents for flood inundation mapping , 2016 .

[13]  Wang Yu-x.  Urban Waterlogging Monitoring and Warning System Based on GIS and RS , 2013 .

[14]  Iman Saleh,et al.  Social-Network-Sourced Big Data Analytics , 2013, IEEE Internet Computing.

[15]  Jiajun Zhang,et al.  Deep Neural Networks in Machine Translation: An Overview , 2015, IEEE Intelligent Systems.

[16]  Kayoko Yamamoto,et al.  Development of Social Media GIS to Support Information Utilization from Normal Times to Disaster Outbreak Times , 2015 .

[17]  P. Reichenbach,et al.  Gis Technology in Mapping Landslide Hazard , 1995 .

[18]  Stuart E. Middleton,et al.  Real-Time Crisis Mapping of Natural Disasters Using Social Media , 2014, IEEE Intelligent Systems.

[19]  P. Frazier,et al.  Water body detection and delineation with Landsat TM data. , 2000 .

[20]  A. Dhakal,et al.  Detection of areas associated with flood and erosion caused by a heavy rainfall using multitemporal Landsat TM data , 2002 .

[21]  Min Li,et al.  Sparse and Low-Rank Coupling Image Segmentation Model Via Nonconvex Regularization , 2015, Int. J. Pattern Recognit. Artif. Intell..

[22]  M. Campagna,et al.  The Role of Social Media Geographic Information (SMGI) in Spatial Planning , 2015 .

[23]  Deanne Bird,et al.  Flooding Facebook - the use of social media during the Queensland and Victorian floods , 2012 .

[24]  Lei Guo,et al.  Effective and Efficient Midlevel Visual Elements-Oriented Land-Use Classification Using VHR Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Xing Xie,et al.  User-Service Rating Prediction by Exploring Social Users' Rating Behaviors , 2016, IEEE Transactions on Multimedia.

[26]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[27]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[28]  Yong Wang,et al.  An efficient method for mapping flood extent in a coastal floodplain using Landsat TM and DEM data , 2002 .

[29]  Damien Serre,et al.  Critical Data Source; Tool or Even Infrastructure? Challenges of Geographic Information Systems and Remote Sensing for Disaster Risk Governance , 2015, ISPRS Int. J. Geo Inf..

[30]  Vijayan Sugumaran,et al.  Crowdsourcing based spatial mining of urban emergency events using social media , 2015, EM-GIS.

[31]  Paul D. Bates,et al.  Automatic near real-time selection of flood water levels from high resolution Synthetic Aperture Radar images for assimilation into hydraulic models: A case study , 2012 .

[32]  Joshua D Werts,et al.  An Integrated WebGIS Framework for Volunteered Geographic Information and Social Media in Soil and Water Conservation , 2012, Environmental Management.

[33]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[34]  John Krumm,et al.  Discovering points of interest from users’ map annotations , 2008 .

[35]  J. C. Knox,et al.  Orbital SAR remote sensing of a river flood wave , 1998 .

[36]  Huajun Chen,et al.  Social Media Meets Big Urban Data: A Case Study of Urban Waterlogging Analysis , 2016, Comput. Intell. Neurosci..

[37]  Christopher Cheong,et al.  Social Media Data Mining: A Social Network Analysis Of Tweets During The 2010-2011 Australian Floods , 2011, PACIS.

[38]  S. Natarajan,et al.  How social media can contribute during disaster events? Case study of Chennai floods 2015 , 2016, 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[39]  Michelle R. Guy,et al.  Twitter earthquake detection: earthquake monitoring in a social world , 2012 .

[40]  Ching Y. Suen,et al.  Detecting predatory conversations in social media by deep Convolutional Neural Networks , 2016, Digit. Investig..

[41]  Zhihui Lai,et al.  Manifold discriminant regression learning for image classification , 2015, Neurocomputing.

[42]  Kathleen M. Carley,et al.  Using tweets to support disaster planning, warning and response , 2016 .

[43]  Vijayan Sugumaran,et al.  Participatory sensing-based semantic and spatial analysis of urban emergency events using mobile social media , 2016, EURASIP J. Wirel. Commun. Netw..

[44]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[45]  Gian Luca Foresti,et al.  Situational awareness in smart environments: socio-mobile and sensor data fusion for emergency response to disasters , 2015, J. Ambient Intell. Humaniz. Comput..

[46]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[47]  Junwei Han,et al.  A Survey on Object Detection in Optical Remote Sensing Images , 2016, ArXiv.

[48]  Yutaka Matsuo,et al.  Tweet Analysis for Real-Time Event Detection and Earthquake Reporting System Development , 2013, IEEE Transactions on Knowledge and Data Engineering.

[49]  P. Bates.  Integrating remote sensing data with flood inundation models: how far have we got? , 2012 .

[50]  Hui Zhang,et al.  Semantic analysis on microblog data for emergency response in typhoon Chan-hom , 2015, EM-GIS.

[51]  P. Bates,et al.  Near real time satellite imagery to support and verify timely flood modelling , 2009 .

[52]  Edilson de Aguiar,et al.  Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order , 2017, Pattern Recognit..

[53]  Ferda Ofli,et al.  Combining Human Computing and Machine Learning to Make Sense of Big (Aerial) Data for Disaster Response , 2016, Big Data.

[54]  Alexander Zipf,et al.  Exploring the Geographical Relations Between Social Media and Flood Phenomena to Improve Situational Awareness - A Study About the River Elbe Flood in June 2013 , 2014, AGILE Conf..

[55]  Basit Shafiq,et al.  Spatial Computing and Social Media in the Context of Disaster Management , 2012, IEEE Intelligent Systems.

[56]  Xuelong Li,et al.  Semi-Supervised Multitask Learning for Scene Recognition , 2015, IEEE Transactions on Cybernetics.

[57]  Martha Grabowski,et al.  Social Media and Warning Response Impacts in Extreme Events: Results from a Naturally Occurring Experiment , 2012, 2012 45th Hawaii International Conference on System Sciences.

[58]  Anthony Stefanidis,et al.  #Earthquake: Twitter as a Distributed Sensor System , 2013, Trans. GIS.

[59]  Kyumin Lee,et al.  You are where you tweet: a content-based approach to geo-locating twitter users , 2010, CIKM.

[60]  Andreas Dengel,et al.  Contextual Enrichment of Remote-Sensed Events with Social Media Streams , 2016, ACM Multimedia.

[61]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[62]  Leysia Palen,et al.  Mastering social media: An analysis of Jefferson County's communications during the 2013 Colorado floods , 2014, ISCRAM.

[63]  Helen L. Partridge,et al.  Exploring information experience using social media during the 2011 Queensland Floods: a pilot study , 2012 .

[64]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[65]  David S. Ebert,et al.  Spatiotemporal social media analytics for abnormal event detection and examination using seasonal-trend decomposition , 2012, 2012 IEEE Conference on Visual Analytics Science and Technology (VAST).

[66]  Xuelong Li,et al.  Latent Semantic Minimal Hashing for Image Retrieval , 2017, IEEE Transactions on Image Processing.

[67]  Bertrand De Longueville,et al.  "OMG, from here, I can see the flames!": a use case of mining location based social networks to acquire spatio-temporal data on forest fires , 2009, LBSN '09.

[68]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[69]  R. Colombo,et al.  Integration of remote sensing data and GIS for accurate mapping of flooded areas , 2002 .

[70]  Zongtian Liu,et al.  Event Recognition Based on Deep Learning in Chinese Texts , 2016, PloS one.

[71]  Daniel Dajun Zeng,et al.  Mining opinion summarizations using convolutional neural networks in Chinese microblogging systems , 2016, Knowl. Based Syst..

[72]  Kayoko Yamamoto.  Development and Operation of Social Media GIS for Disaster Risk Management in Japan , 2015 .

[73]  K. Field,et al.  Cartoblography: Experiments in Using and Organising the Spatial Context of Micro‐blogging , 2010 .

[74]  Sukeshini A. Grandhi,et al.  P3 systems: putting the place back into social networks , 2005, IEEE Internet Computing.

[75]  Michael F. Goodchild,et al.  GEDMWA: geospatial exploratory data mining web agent , 2008, GIS '08.

[76]  Yong Wang,et al.  Advances in Remote Sensing of Flooding , 2015 .

[77]  Jing Ma,et al.  The socio-spatial dimension of behavior analysis: Frontiers and progress in Chinese behavioral geography , 2016, Journal of Geographical Sciences.

[78]  Lei Guo,et al.  Object Detection in Optical Remote Sensing Images Based on Weakly Supervised Learning and High-Level Feature Learning , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[79]  Yutaka Matsuo,et al.  Semantic Twitter: Analyzing Tweets for Real-Time Event Notification , 2008, BlogTalk.

[80]  Luke de Oliveira,et al.  Image Processing, Computer Vision, and Deep Learning: new approaches to the analysis and physics interpretation of LHC events , 2016, Journal of Physics: Conference Series.

[81]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[82]  Qunying Huang,et al.  Understanding social media data for disaster management , 2015, Natural Hazards.

[83]  Huajun Chen,et al.  Monitoring Urban Waterlogging Disaster Using Social Sensors , 2014, CSWS.

[84]  O. K. Gowrishankar,et al.  Personalized Travel Sequence Recommendation on Multi-Source Big Social Media , 2016, IEEE Transactions on Big Data.

[85]  Keith W. Miller,et al.  Big Data: New Opportunities and New Challenges [Guest editors' introduction] , 2013, Computer.

[86]  Adam Acar,et al.  Twitter for crisis communication: lessons learned from Japan's tsunami disaster , 2011, Int. J. Web Based Communities.

[87]  Emily Schnebele,et al.  Improving remote sensing flood assessment using volunteered geographical data , 2013 .

[88]  Adam L. Berger,et al.  A Maximum Entropy Approach to Natural Language Processing , 1996, CL.

[89]  A. Stefanidis,et al.  Harvesting ambient geospatial information from social media feeds , 2011, GeoJournal.

[90]  Angela Olasz,et al.  Using big geospatial data for fast flood detection: Developments from the IQmulus project , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[91]  Jin Li,et al.  Spatial interpolation methods applied in the environmental sciences: A review , 2014, Environ. Model. Softw..

[92]  M. Clark,et al.  Putting water in its place: a perspective on GIS in hydrology and water management , 1998 .

[93]  Jian Bai,et al.  Image decomposition combining a total variational filter and a Tikhonov quadratic filter , 2015, Multidimens. Syst. Signal Process..

[94]  Konrad Paul Kording,et al.  Fall Classification by Machine Learning Using Mobile Phones , 2012, PloS one.

[95]  J Brian Houston,et al.  Social media and disasters: a functional framework for social media use in disaster planning, response, and research. , 2015, Disasters.

[96]  Michael F. Goodchild,et al.  The convergence of GIS and social media: challenges for GIScience , 2011, Int. J. Geogr. Inf. Sci..

[97]  Lars Backstrom,et al.  Find me if you can: improving geographical prediction with social and spatial proximity , 2010, WWW '10.

[98]  Norman M. Sadeh,et al.  The Livehoods Project: Utilizing Social Media to Understand the Dynamics of a City , 2012, ICWSM.

[99]  P. Earle,et al.  OMG Earthquake! Can Twitter Improve Earthquake Response? , 2009 .

[100]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[101]  Xiao Liu,et al.  Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset , 2016, Neurocomputing.

[102]  M. Goodchild,et al.  Researching Volunteered Geographic Information: Spatial Data, Geographic Research, and New Social Practice , 2012 .

[103]  Kathleen M. Carley,et al.  Crowd sourcing disaster management: The complex nature of Twitter usage in Padang Indonesia , 2016 .

[104]  LinLin Shen,et al.  3D Gabor wavelets for evaluating SPM normalization algorithm , 2008, Medical Image Anal..

[105]  Michael F. Goodchild,et al.  Please Scroll down for Article International Journal of Digital Earth Crowdsourcing Geographic Information for Disaster Response: a Research Frontier Crowdsourcing Geographic Information for Disaster Response: a Research Frontier , 2022 .

[106]  George Baciu,et al.  Lightness biased cartoon-and-texture decomposition for textile image segmentation , 2015, Neurocomputing.

[107]  J. A. Quintanilha,et al.  Flooding and inundation collaborative mapping – use of the Crowdmap/Ushahidi platform in the city of Sao Paulo, Brazil , 2018 .

[108]  Heidi Kreibich,et al.  Social media as an information source for rapid flood inundation mapping , 2015 .

[109]  Tao Mei,et al.  Service Quality Evaluation by Exploring Social Users’ Contextual Information , 2016, IEEE Transactions on Knowledge and Data Engineering.

[110]  Rui Li,et al.  TEDAS: A Twitter-based Event Detection and Analysis System , 2012, 2012 IEEE 28th International Conference on Data Engineering.

[111]  D. Dransch,et al.  Volunteered Geographic Information for Disaster Management with Application to Rapid Flood Damage Estimation , 2019 .

[112]  Jie Yin,et al.  Using Social Media to Enhance Emergency Situation Awareness , 2012, IEEE Intelligent Systems.

[113]  Geert-Jan Houben,et al.  Semantics + filtering + search = twitcident. exploring information in social web streams , 2012, HT '12.

[114]  Daniel A. Keim,et al.  Visual Analysis of Social Media Data , 2013, Computer.

[115]  Anthony Stefanidis,et al.  Geosocial gauge: a system prototype for knowledge discovery from social media , 2013, Int. J. Geogr. Inf. Sci..

[116]  Xiaoqiang Lu,et al.  Scene Recognition by Manifold Regularized Deep Learning Architecture , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[117]  Cecilia Mascolo,et al.  Socio-Spatial Properties of Online Location-Based Social Networks , 2011, ICWSM.

[118]  Xuelong Li,et al.  Manifold Regularized Sparse NMF for Hyperspectral Unmixing , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[119]  Hyun Myung,et al.  Weighted joint-based human behavior recognition algorithm using only depth information for low-cost intelligent video-surveillance system , 2016, Expert Syst. Appl..

[120]  Miriam J. Metzger,et al.  The credibility of volunteered geographic information , 2008 .

[121]  Xueming Qian,et al.  Semantic Annotation of High-Resolution Satellite Images via Weakly Supervised Learning , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[122]  Sarah Vieweg,et al.  Processing Social Media Messages in Mass Emergency , 2014, ACM Comput. Surv..

[123]  Shiyuan Xu,et al.  Community-based scenario modelling and disaster risk assessment of urban rainstorm waterlogging , 2011 .