Content-Based Multimedia Retrieval

With rapid advances in storage devices, networks, and compression techniques, large-scale multimedia data has become available to average users. How to index and search multimedia data according to its real content is a challenging problem, which has been studied for decades. Since the 1990s, content-based multimedia retrieval (CBMR) had become an increasingly active field, which is defined as searching for desired multimedia data (images or video/audio segments/clips) relevant with issued queries, such as image/audio/video examples, keywords, phrases, sentences, or any combination of them. Different from text retrieval, CBMR is a more challenging task, as certain understanding of the content of multimedia data. It mainly involves two basic problems. One problem is how to represent queries and multimedia content. The other problem is how to map the representations of queries and multimedia content. Extensive works on CBMR have already been published, and different paradigms and techniques have been proposed, such as query-by-example (QBE), annotation-based retrieval (ABR), and multimodality retrieval. Here we conclude that they are all proposed aiming at addressing the above two issues. QBE is the most typical scenario for multimedia retrieval before the year of 2000, whereas query by text and by combination of text and examples have become two new mainstream scenarios thereafter. In this article, we will introduce both classic CBMR and the current mainstream. Keywords: multimedia retrieval; multimedia search; relevance feedback; multimedia database; pattern recognition; multimedia content analysis; video; image