LSCOM Lexicon Definitions and Annotations (Version 1.0)

The DTO sponsored LSCOM workshop is developing an expanded multimedia concept lexicon on the order of 1000. Concepts related to events, objects, locations, people, and programs have been selected following a multi-step process involving input solicitation, expert critiquing, comparison with related ontologies, and performance evaluation. Participants of the process include representatives from intelligence community users, ontology specialists, and multimedia analytics researchers. In addition, each concept has been qualitatively assessed according to some criteria, such as utility (usefulness), observability (by humans), and feasibility (by automatic detection). An annotation process was completed in late 2005 by student annotators at Columbia University and CMU, over the entire development set of TRECVID 2005 videos. Human subjects judge the presence or absence of each concept in the key frame of each subshot, resulting in a total of 61901 labels for each concept. Out of the 834 initial selected concepts, the first version of LSCOM annotations consists of 449 unique concepts (39 LSCOM-Lite concepts included) over the entire TRECVID 2005 development set (61901 subshots). The LSCOM-Lite annotations [1] include 39 high-level features (concepts), which are interim results from the effort in developing a Large-Scale Concept Ontology for Multimedia (LSCOM). The concepts were selected based on semi-automatic mapping of 26377 noun search terms from BBC query logs in late 1998 to WordNet senses, division of semantic concept space into a small number of orthogonal dimensions, and evaluation of 2003 and 2004 TRECVID search topics. The dimensions consist of program category, setting/scene/site, people, object, activity, event, and graphics. A collaborative effort was 2 completed in 2005 to produce annotations of the 39 concepts over the entire development set of TRECVID 2005 videos. Human subjects judge the presence or absence of each concept in the key frame of each subshot, resulting in a total of 61901 labels for each concept. Ten of the LSCOM-Lite concepts have been chosen for evaluation in TRECVID 2005 high-level feature detection task [2]. Appendix A shows the listing and definitions for the 856 concepts included in LSCOM Lexicon Version 1.