Gathering training sample automatically for social event visual modeling

In recent years, the emergence of social media on the Internet has derived many of interesting research and applications. In this paper, a novel framework is proposed to model the visual appearance of social events using automatically collected training samples on the basis of photo context analysis. While collecting positive samples can be achieved easily thanks to explicitly identifying tags, finding representative negative samples from the vast amount of irrelevant multimedia documents is a more challenging task. Here, we argue and demonstrate that the most common negative sample, originating from the same location as the event to be modeled, are best suited for the task. A novel ranking approach is devised to select a set of negative samples. The visual event models are learned from automatically collected samples using SVM. The results reported here show that the event models are effective to filter out irrelevant photos and perform with a high accuracy on various social events categories.

[1]  Wolfgang Nejdl,et al.  Bringing order to your photos: event-driven classification of flickr images based on social knowledge , 2010, CIKM.

[2]  Tat-Seng Chua,et al.  NUS-WIDE: a real-world web image database from National University of Singapore , 2009, CIVR '09.

[3]  Luc Van Gool,et al.  World-scale mining of objects and events from community photo collections , 2008, CIVR '08.

[4]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[5]  Ramesh Jain,et al.  Toward a Common Event Model for Multimedia Applications , 2007, IEEE MultiMedia.

[6]  Hila Becker,et al.  Event Identification in Social Media , 2009, WebDB.

[7]  Raphaël Troncy,et al.  Finding media illustrating events , 2011, ICMR '11.

[8]  Antonio Criminisi,et al.  Harvesting Image Databases from the Web , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Jianping Fan,et al.  Harvesting large-scale weakly-tagged image databases from the web , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Marcel Worring,et al.  Social negative bootstrapping for visual categorization , 2011, ICMR '11.

[11]  João Magalhães,et al.  Automated Illustration of News Stories , 2010, 2010 IEEE Fourth International Conference on Semantic Computing.

[12]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[13]  Paul Over,et al.  TRECVID 2008 - Goals, Tasks, Data, Evaluation Mechanisms and Metrics , 2010, TRECVID.

[14]  Fei-Fei Li,et al.  OPTIMOL: Automatic Online Picture Collection via Incremental Model Learning , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Tat-Seng Chua,et al.  Exploring large scale data for multimedia QA: an initial study , 2010, CIVR '10.

[16]  Hila Becker,et al.  Learning similarity metrics for event identification in social media , 2010, WSDM '10.

[17]  Gang Wang,et al.  On the sampling of web images for learning visual concept classifiers , 2010, CIVR '10.

[18]  Hila Becker,et al.  Identifying content for planned events across social media sites , 2012, WSDM '12.

[19]  Tao Mei,et al.  Graph-based semi-supervised learning with multi-label , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[20]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.