TRECVID 2004 - An Overview

TRECVID 2004 was the fourth running of a TRECstyle video retrieval evaluation, the goal of which remains to promote progress in content-based retrieval from digital video via open, metrics-based evaluation. Over time this effort should yield a better understanding of how systems can effectively accomplish such retrieval and how one can reliably benchmark their performance. TRECVID is funded by the Advanced Research and Development Activity (ARDA) and the National Institute of Standards and Technology (NIST). The evaluation used as test data about 70 hours of US broadcast news video in MPEG-1 format that had been collected for TDT-2 by the Linguistic Data Consortium in 1998. 33 teams from various research organizations — 7 from Asia/Australia, 17 from Europe, and 9 from the Americas — participated in one or more of four tasks: shot boundary determination, story segmentation, feature extraction, and search (manual or interactive). Results were scored by NIST using manually created truth data for shot boundary determination and story segmentation. Feature extraction and search submissions were evaluated based on partial manual judgments of the pooled submissions. This paper is an introduction to, and an overview of, the evaluation framework (the tasks, data, and measures), the results, and the approaches taken by the participating groups. For detailed information about the approaches and results, the reader should see the online proceedings on the TRECVID website (www-nlpir.nist.gov/projects/trecvid).