Multimodal Salient Objects: General Building Blocks of Semantic Video Concepts

In this paper, we propose a novel video content representation framework to achieve a middle-level understanding of video contents by using multimodal salient objects. Specifically, this framework includes: (a) A semantic-sensitive video content representation and semantic video concept modeling framework by using the multimodal salient objects and Gaussian mixture models; (b) A machine learning technique to train the automatic detection functions of multimodal salient objects; (c) A novel framework to enable more effective classifier training by integrating model selection and parameter estimation seamlessly in a single algorithm. Our experiments on a certain domain of medical education videos have obtained very convincing results.