TRECVID 2004 Search and Feature Extraction Task by NUS PRIS

This paper describes the details of our systems for feature extraction and search tasks of TRECVID-2004. For feature extraction, we emphasize the use of visual auto-concept annotation technique, with the fusion of text and specialized detectors, to induce concepts in videos. For the search task, our emphasis is two-fold. First we employ query-specific models, and second, we employ multi-modality features, including text, annotated visual concepts, OCR output, shot classes and specialized detectors to perform the search. Our search pipeline is similar to that employed in text-based definition question-answering approaches. We first perform query analysis to categorize the query into the categories of: {PERSON, SPORTS, FINANCE, WEATHER, DISASTER and GENERAL}. From these categories, we induce a number of constraints on the search process, including: (a) the type of multi-modality features to use or emphasize; (b) the key concept terms in text query to use; and (c) the video classes, such as sports or anchor person etc to use or exclude in the search results. The results on 60 hours of test video from TRECVID 2004 evaluation demonstrate that our approaches are effective.