Subspace-based Semantic Concept Detection and Retrieval for Multimedia Information Systems

of a dissertation at the University of Miami. Dissertation supervised by Professor Mei-Ling Shyu No. of pages in text. (170) The prevalence of digital recording devices, the cheap cost of data storage as well as the convenience provided by the widely accessible Internet have created the demand to retrieve information according to users’ requests from multimedia data sources. However, the multimedia information retrieval task has several challenges that need to be addressed, such as bridging the semantic gap, modeling from imbalanced data sets, and utilizing inter-concept relationships to enhance the retrieval performance of an individual concept. To respond to the challenge of bridging the semantic gap, subspace modeling methods are proposed to address this issue as a classification task. The proposed subspace modeling methods construct a principal component (PC) subspace for each class, where the PCs are derived from the instances belonging to that class. The PCs are selected and ranked based on Fisher’s criterion to reduce the searching effort and an iterative searching is utilized to determine the best PC set. Subspace modeling methods are proposed in this dissertation, including multi-class subspace modeling (MSM), binary-class subspace modeling (BSM), and subspace modeling on global and local structures (SMGL). Comparative experiments show that MSM, BSM, and SMGL can outperform some other well-known algorithms on a number of benchmark data sets. To address the data imbalance challenge, two clustering-based subspace modeling methods called clustering-based subspace modeling (CLU-SUMO) and class selection and clusteringbased subspace modeling (CSC-SUMO) are proposed. K-means clustering and/or semantic concept labels are used to partition the majority class (usually the negative class) into several groups, each of which is merged with the minority class (usually the positive class) to form a much more balanced subset of the original data set. Then, the subspace model learned from the original data set is integrated with all the subspace models learned from the balanced subsets to form a classification framework. The experimental results on news and broadcast video data sets support the claim that the proposed CLU-SUMO and CSC-SUMO render better classification performance than some existing techniques that are commonly used to handle the data imbalance