With rapid advances in storage devices, networks, and compression techniques, large-scale multimedia data has become available to average users. How to index and search multimedia data according to its real content is a challenging problem, which has been studied for decades. Since the 1990s, content-based multimedia retrieval (CBMR) had become an increasingly active field, which is defined as searching for desired multimedia data (images or video/audio segments/clips) relevant with issued queries, such as image/audio/video examples, keywords, phrases, sentences, or any combination of them.
Different from text retrieval, CBMR is a more challenging task, as certain understanding of the content of multimedia data. It mainly involves two basic problems. One problem is how to represent queries and multimedia content. The other problem is how to map the representations of queries and multimedia content. Extensive works on CBMR have already been published, and different paradigms and techniques have been proposed, such as query-by-example (QBE), annotation-based retrieval (ABR), and multimodality retrieval. Here we conclude that they are all proposed aiming at addressing the above two issues. QBE is the most typical scenario for multimedia retrieval before the year of 2000, whereas query by text and by combination of text and examples have become two new mainstream scenarios thereafter. In this article, we will introduce both classic CBMR and the current mainstream.
Keywords:
multimedia retrieval;
multimedia search;
relevance feedback;
multimedia database;
pattern recognition;
multimedia content analysis;
video;
image
[1]
Marcel Worring,et al.
Content-Based Image Retrieval at the End of the Early Years
,
2000,
IEEE Trans. Pattern Anal. Mach. Intell..
[2]
Markus A. Stricker,et al.
Similarity of color images
,
1995,
Electronic Imaging.
[3]
Ying Liu,et al.
A survey of content-based image retrieval with high-level semantics
,
2007,
Pattern Recognit..
[4]
Shih-Fu Chang,et al.
Reranking Methods for Visual Search
,
2007,
IEEE MultiMedia.
[5]
Shih-Fu Chang,et al.
Image Retrieval: Current Techniques, Promising Directions, and Open Issues
,
1999,
J. Vis. Commun. Image Represent..
[6]
Atsuo Yoshitaka,et al.
A Survey on Content-Based Retrieval for Multimedia Databases
,
1999,
IEEE Trans. Knowl. Data Eng..
[7]
Vijay V. Raghavan,et al.
Content-Based Image Retrieval Systems - Guest Editors' Introduction
,
1995,
Computer.
[8]
Jonathan Foote,et al.
An overview of audio information retrieval
,
1999,
Multimedia Systems.
[9]
Goujun Lu,et al.
Indexing and Retrieval of Audio: A Survey
,
2001,
Multimedia Tools and Applications.