Moderate Vocabulary Visual Concept Detection for the TRECVID 2002

The explosion in multimodal content availability underlines the necessity for content management at a semantic level. We have cast the problem of detecting semantics in multimedia content as a pattern classification problem and the problem of building models of multimodal semantics as a learning problem. Recent trends show increasing use of statistical machine learning providing a computational framework for mapping low level media features to high level semantic concepts. In this chapter we expose the challenges that these techniques face. We show that if a lexicon of visual concepts is identified a priori, a statistical framework can be used to build visual feature models for the concepts in the lexicon. Using support vector machine (SVM) classification we build models for 34 semantic concepts for the TREC 2002 benchmark corpus. We study the effect of number of examples available for training with respect to their impact on detection. We also examine low level feature fusion as well as parameter sensitivity with SVM classifiers.