A multiple kernel learning based fusion for earthquake detection from multimedia twitter data

An efficient way of extracting useful information from multiple sources of data is to use data fusion technology. This paper introduces a data fusion approach in multimedia data for earthquake detection in twitter by using kernel fusion. The fusion method applies to fuse two types of data. The first type is features extracted from text by using bag-of-words method which is based on the calculation of the term frequency-inverse document frequency. The second type is the visual features extracted from images by applying scale-invariant feature transform. A multiple kernel fusion is applied in order to fuse the information from these two sources. Our experiments have indicated that comparing to the approaches using single data source, the proposed approach of using multiple kernel learning algorithm as early fusion increased the accuracy for earthquake detection. Experimental results for the proposed method achieved a high accuracy of 0.94, comparing to accuracy of 0.89 with texts only, and accuracy of 0.83 with images only.

[1]  Nishchol Mishra,et al.  Image Mining in the Context of Content Based Image Retrieval: A Perspective , 2012 .

[2]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[3]  Lawrence A. Klein,et al.  Sensor and Data Fusion: A Tool for Information Assessment and Decision Making , 2004 .

[4]  Takahiro Kawamura,et al.  Building an earthquake evacuation ontology from twitter , 2011, 2011 IEEE International Conference on Granular Computing.

[5]  Francesco G. B. De Natale,et al.  Event Clustering and Classification from Social Media: Watershed-based and Kernel Methods , 2013, MediaEval.

[6]  Pinar Senkul,et al.  Semantic Expansion of Hashtags for Enhanced Event Detection in Twitter , 2012 .

[7]  Sabalan Daneshvar,et al.  MRI and PET image fusion by combining IHS and retina-inspired models , 2010, Inf. Fusion.

[8]  Ling Guan,et al.  Multimodal information fusion of audiovisual emotion recognition using novel information theoretic tools , 2013, ICME.

[9]  Xiaoming Zhang,et al.  Event detection and popularity prediction in microblogging , 2015, Neurocomputing.

[10]  Jingxian Mao.  Multimodal Data Fusion As a Predictior of Missing Information in Social Networks , 2012 .

[11]  Ruofei Zhang,et al.  Multimedia Data Mining: A Systematic Introduction to Concepts and Theory , 2008 .

[12]  Daniel P. Huttenlocher,et al.  Landmark classification in large-scale image collections , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[13]  Wei Liu,et al.  Multimedia classification and event detection using double fusion , 2013, Multimedia Tools and Applications.

[14]  Georgios Petkos,et al.  Social Event Detection at MediaEval : a three-year retrospect of tasks and results , 2014 .

[15]  Brian Regan,et al.  Fusing Text and Image for Event Detection in Twitter , 2015, ArXiv.

[16]  Pingzhi Fan,et al.  Performance evaluation of score level fusion in multimodal biometric systems , 2010, Pattern Recognit..

[17]  Deanna Needell,et al.  Improving image clustering using sparse text and the wisdom of the crowds , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[18]  Jesse Harriott,et al.  Win with Advanced Business Analytics: Creating Business Value from Your Data , 2012 .

[19]  Chong-Wah Ngo,et al.  Evaluating bag-of-visual-words representations in scene classification , 2007, MIR '07.

[20]  Shangbo Zhou,et al.  Multimedia Data Fusion , 2013 .

[21]  Danah Boyd,et al.  Tweeting from the Town Square: Measuring Geographic Local Networks , 2010, ICWSM.

[22]  Wael Khreich,et al.  A Survey of Techniques for Event Detection in Twitter , 2015, Comput. Intell..

[23]  Michelle R. Guy,et al.  Twitter earthquake detection: earthquake monitoring in a social world , 2012 .

[24]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[25]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[26]  Anthony Stefanidis,et al.  #Earthquake: Twitter as a Distributed Sensor System , 2013, Trans. GIS.

[27]  Timothy E. Dawson,et al.  The Mw 6.0 24 August 2014 South Napa Earthquake , 2015 .

[28]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[29]  Vasileios Mezaris.  Social event detection at MediaEval: a 3-year retrospect of tasks and results , 2014 .

[30]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[31]  Yiannis Kompatsiaris,et al.  Semantic Multimedia and Ontologies , 2008 .

[32]  Barbara Poblete,et al.  Twitter under crisis: can we trust what we RT? , 2010, SOMA '10.

[33]  Mohan S. Kankanhalli,et al.  Modeling, detecting, and processing events in multimedia , 2010, ACM Multimedia.

[34]  L. Burks,et al.  RAPID ESTIMATE OF GROUND SHAKING INTENSITY BY COMBINING SIMPLE EARTHQUAKE CHARACTERISTICS WITH TWEETS , 2014 .

[35]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[36]  Halit Oguztüzün,et al.  Semantic Expansion of Tweet Contents for Enhanced Event Detection in Twitter , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[37]  Tzung-Pei Hong,et al.  Social Network Mining, Analysis and Research Trends: Techniques and Applications , 2011 .

[38]  Reza Zafarani,et al.  Social Media Mining: An Introduction , 2014 .

[39]  Sung Wook Baik,et al.  Visual saliency models for summarization of diagnostic hysteroscopy videos in healthcare systems , 2016, SpringerPlus.