Finding media illustrating events

We present a method combining semantic inferencing and visual analysis for finding automatically media (photos and videos) illustrating events. We report on experiments validating our heuristic for mining media sharing platforms and large event directories in order to mutually enrich the descriptions of the content they host. Our overall goal is to design a web-based environment that allows users to explore and select events, to inspect associated media, and to discover meaningful, surprising or entertaining connections between events, media and people participating in events. We present a large dataset composed of semantic descriptions of events, photos and videos interlinked with the larger Linked Open Data cloud and we show the benefits of using semantic web technologies for integrating multimedia metadata.

[1]  Mark B. Sandler,et al.  The Music Ontology , 2007, ISMIR.

[2]  Ramesh Jain,et al.  Toward a Common Event Model for Multimedia Applications , 2007, IEEE MultiMedia.

[3]  André Fialho.  What's on this evening ? Designing user support for event-based annotation and exploration of media , 2010 .

[4]  Gerben de Vries,et al.  Combining ship trajectories and semantics with the simple event model (SEM) , 2009, EiMM '09.

[5]  Yang Song,et al.  Tour the world: Building a web-scale landmark recognition engine , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Raphaël Troncy,et al.  Linking events with media , 2010, I-SEMANTICS '10.

[7]  Mor Naaman,et al.  Less talk, more rock: automated organization of community-contributed collections of concert videos , 2009, WWW '09.

[8]  Takahiro Hara,et al.  Mining people's trips from large scale geo-tagged photos , 2010, ACM Multimedia.

[9]  Raphaël Troncy,et al.  Experiencing Events through User-Generated Media , 2010, COLD.

[10]  Raphaël Troncy,et al.  LODE: Linking Open Descriptions of Events , 2009, ASWC.

[11]  Dong Liu,et al.  Image retagging , 2010, ACM Multimedia.

[12]  Hila Becker,et al.  Event Identification in Social Media , 2009, WebDB.

[13]  Shuicheng Yan,et al.  Inferring semantic concepts from community-contributed images and noisy tags , 2009, ACM Multimedia.

[14]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[15]  Hila Becker,et al.  Learning similarity metrics for event identification in social media , 2010, WSDM '10.

[16]  Marti A. Hearst.  Search User Interfaces , 2009 .