I-SEARCH: A Unified Framework for Multimodal Search and Retrieval

In this article, a unified framework for multimodal search and retrieval is introduced. The framework is an outcome of the research that took place within the I-SEARCH European Project. The proposed system covers all aspects of a search and retrieval process, namely low-level descriptor extraction, indexing, query formulation, retrieval and visualisation of the search results. All I-SEARCH components advance the state of the art in the corresponding scientific fields. The I-SEARCH multimodal search engine is dynamically adapted to end-user's devices, which can vary from a simple mobile phone to a high-performance PC.

[1]  Hong Zhang,et al.  Measuring Multi-modality Similarities Via Subspace Learning for Cross-Media Retrieval , 2006, PCM.

[2]  Avi Arampatzis,et al.  www.MMRetrieval.net: a multimodal search engine , 2010, SISAP.

[3]  Benjamin B. Bederson,et al.  PhotoMesa: a zoomable image browser using quantum treemaps and bubblemaps , 2001, UIST '01.

[4]  Petros Daras,et al.  Introducing a unified framework for content object description , 2011, Int. J. Multim. Intell. Secur..

[5]  Matthieu Guillaumin,et al.  Combining Image-Level and Segment-Level Models for Automatic Annotation , 2012, MMM.

[6]  Petros Daras,et al.  I-SEARCH: a multimodal search engine based on rich unified content description (RUCoD) , 2012, WWW.

[7]  Yi Yang,et al.  Ranking with local regression and global alignment for cross media retrieval , 2009, ACM Multimedia.

[8]  Stuart Macdonald,et al.  User Engagement in Research Data Curation , 2009, ECDL.

[9]  Petros Daras,et al.  Multimodal search and retrieval using manifold learning and query formulation , 2011, Web3D '11.

[10]  Andreas Paepcke,et al.  Time as essence for photo browsing through personal digital libraries , 2002, JCDL '02.

[11]  Mary Czerwinski,et al.  PhotoTOC: automatic clustering for browsing personal photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[12]  Claudio Gennaro,et al.  An Approach to Content-Based Image Retrieval Based on the Lucene Search Engine Library , 2010, ECDL.

[13]  Paul Grimm,et al.  Context-Aware Querying for Multimodal Search Engines , 2012, MMM.

[14]  Petros Daras,et al.  Optimizing Multimedia Retrieval Using Multimodal Fusion and Relevance Feedback Techniques , 2012, MMM.

[15]  Antonio Camurri,et al.  A System for Real-Time Multimodal Analysis of Nonverbal Affective Social Interaction in User-Centric Media , 2010, IEEE Transactions on Multimedia.

[16]  Zagoris.  MMRetrieval . net : a multimodal search engine , .

[17]  Petros Daras,et al.  Search and Retrieval of Rich Media Objects Supporting Multiple Multimodal Queries , 2012, IEEE Transactions on Multimedia.

[18]  Antonio Camurri,et al.  Toward a Minimal Representation of Affective Gestures , 2011, IEEE Transactions on Affective Computing.

[19]  Kiyoharu Aizawa,et al.  Advances in Multimedia Information Processing - PCM 2004, 5th Pacific Rim Conference on Multimedia, Tokyo, Japan, November 30 - December 3, 2004, Proceedings, Part I , 2005, Pacific Rim Conference on Multimedia.

[20]  Petros Daras,et al.  A Future Perspective on the 3D Media Internet , 2009, Future Internet Assembly.