Multimedia retrieval by regions, concepts, and constraints

This thesis investigates the problem of searching multimedia data. In particular, we identify poor retrieval quality and usability as two of the main hurdles for adoption of multimedia search technology. We therefore try to address the above issues by introducing three different multimedia query models and proposing the necessary infrastructure for supporting them. Specifically, we propose methods for region-based, concept-based, and constraint-based retrieval of images. In order to improve retrieval effectiveness, we first propose a similarity model that compares images at the granularity of image objects rather than entire images. We design an efficient algorithm to support the above similarity model and to be robust with respect to scaling and translation of image objects. We also introduce an intermediate layer on top of the query engine that automatically translates simple text queries into complex content-based queries. This concept-based query interface makes the content-based query transparent to the user and improves the system's ease of use. Finally, we consider the problem of matching arbitrary relationships among multimedia objects (e.g., spatial, temporal, and scale constraints). In the second part of the thesis, we study the infrastructure needed for supporting the above query models. More precisely, we consider the problem of efficiently answering top-k nearest neighbor queries. We propose SQL query language extensions and query optimization strategies for search based upon aggregate predicates, which is a generalization of top- k queries. We also investigate top-k queries over joins of ranked inputs. This ordered join problem is fundamental to multimedia query systems but also appears in many traditional database applications. In particular, we propose several algorithms, both exact and approximate, for the ordered join problem, we prove optimality results for them, and we study their behavior empirically.