Multimedia Data Fusion for Event Detection in Twitter by Using Dempster-Shafer Evidence Theory

Data fusion technology can be the best way to extract useful information from multiple sources of data. It has been widely applied in various applications. This paper presents a data fusion approach in multimedia data for event detection in twitter by using Dempster-Shafer evidence theory. The methodology applies a mining algorithm to detect the event. There are two types of data in the fusion. The first is features extracted from text by using the bag-ofwords method which is calculated using the term frequency-inverse document frequency (TF-IDF). The second is the visual features extracted by applying scale-invariant feature transform (SIFT). The Dempster Shafer theory of evidence is applied in order to fuse the information from these two sources. Our experiments have indicated that comparing to the approaches using individual data source, the proposed data fusion approach can increase the prediction accuracy for event detection. The experimental result showed that the proposed method achieved a high accuracy of 0.97, comparing with 0.93 with texts only, and 0.86 with images only. Keywords—Data fusion, Dempster-Shafer theory, data mining, event detection.

[1]  Halit Oguztüzün,et al.  Evidential location estimation for events detected in Twitter , 2013, GIR '13.

[2]  Priyanka Sharma,et al.  Multimodal Classification using Feature Level Fusion and SVM , 2013 .

[3]  Brian Regan,et al.  Fusing Text and Image for Event Detection in Twitter , 2015, ArXiv.

[4]  Ling Guan,et al.  Multimodal information fusion of audiovisual emotion recognition using novel information theoretic tools , 2013, ICME.

[5]  Jesse S. Jin,et al.  Sensor Data Fusion for Accurate Cloud Presence Prediction Using Dempster-Shafer Evidence Theory , 2010, Sensors.

[6]  Shangbo Zhou,et al.  Multimedia Data Fusion , 2013 .

[7]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[8]  Wenyu Zhang,et al.  Novel Paradigm for Constructing Masses in Dempster-Shafer Evidence Theory for Wireless Sensor Network's Multisource Data Fusion , 2014, Sensors.

[9]  Mark T. Maybury,et al.  Multimedia information extraction : papers from the AAAI Fall Symposium , 2008 .

[10]  Lawrence A. Klein,et al.  Sensor and Data Fusion: A Tool for Information Assessment and Decision Making , 2004 .

[11]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[12]  Deanna Needell,et al.  Improving image clustering using sparse text and the wisdom of the crowds , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[13]  Jingxian Mao.  Multimodal Data Fusion As a Predictior of Missing Information in Social Networks , 2012 .

[14]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[15]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[16]  Mohan S. Kankanhalli,et al.  Multimodal fusion for multimedia analysis: a survey , 2010, Multimedia Systems.

[17]  Jesse Harriott,et al.  Win with Advanced Business Analytics: Creating Business Value from Your Data , 2012 .

[18]  Wael Khreich,et al.  A Survey of Techniques for Event Detection in Twitter , 2015, Comput. Intell..

[19]  Yan Jiang,et al.  Data Fusion in Environment Monitoring Systems with Extended Dempster-Shafer Theory , 2014 .

[20]  Nishchol Mishra,et al.  Image Mining in the Context of Content Based Image Retrieval: A Perspective , 2012 .

[21]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.