TRECVID 2006 by NUS-I2R

NUS and IR joint participated in the high-level feature extraction and automated search task for TRECVID 2006. In both task, we only make use of the standard TRECVID available annotation results. For HLF task, we develop 2 methods to perform automated concept annotation: (a) fully machine learning approach using SVM, LDF and GMM; and (b) Bi-gram model for Pattern Discovery and Matching. As for the automated search task, our emphases this year are: 1) integration of HLF: query-analysis to match query to possible related HLF and fuse results from various participating groups in the HLF task; and 2) integration of event structures present implicitly in news video in a time-dependent event-based retrieval. The proposed generic framework involves various multimodal (including HLFs) features as well as the implicit temporal and event structures to support precise news video retrieval. 1. HIGH LEVEL FEATURE EXTRACTION TASK 1.1 Visual Feature Extraction The visual features used in our systems are listed in the below:  Global color correlogram (GCC) in HSV space: 324-dimension.  Co-occurrence texture extracted from global gray-level co-occurrence matrix (GLCM): 64-dimension.  3-D global color histogram in HSV (HSV): 162-dimension.  3-D global color histogram in RGB (RGB): 125-dimension.  3-D global color histogram in LAB (LAB): 125-dimension.  Gabor filter (2-scale, 12 orientations) based texture feature (Gabor): 48-dimension extracted from one of 5*5 patches uniformly segmented of the image. The textual features used are described in Section 2. 1.2 Machine Learning approach by SVM, LDF and GMM For each type of extracted feature, we make use of SVM classifier (SVM), linear discriminative function (LDF) classifier or Gaussian mixture model (GMM) for training. The details are summarized in Table 1. Thus, there are 9 visual classifiers trained for each concept. The SVM is trained using the SVM tool (Joachims, 2002) and the LDF and GMM classifiers are trained using the AUC maximized learning algorithm (Gao & Sun, 2006). Based on our experiments on the development set of TRECVID 2005, we find AUC maximized learning algorithm works better than the best-tuned SVM system. Table 1: Description of classifiers (+: the classifier is available for the feature) SVM LDF GMM GCC + + GLCM + + HSV + + RGB + LAB + Gabor + For each image, we use the 451 (9*39) features from the feature-space and then train a SVM classifier for each concept. PCA or LSA is also applied to reduce the selected feature space dimensions. We follow the approach in (Gao et al, 2007) which uses a knowledge-based method to retain only informative components. This is done by extracting the pair-wise concepts association among the 39 concepts based on the development set. The strength of the association, Str, for the target concept A and the concept B is defined as, * Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China