Concept-oriented video skimming and adaptation via semantic classification

Concept-oriented video skimming and adaptation plays an important role in enabling online medical education by selecting and transmitting the suitable medical video clips to the students over network. In this paper, we propose a novel framework to enable concept-oriented video skimming and adaptation in a specific domain of medical education video. Specifically, this framework includes: (a) A novel semantic-sensitive framework for video content characterization and representation by using principal video shots to enhance the quality of features on discriminating between different semantic video concepts. (b) A novel technique for semantic medical concept interpretation by using finite mixture models to approximate the class distributions of the relevant principal video shots. (c) A novel classifier training scheme by using an adaptive Expectation-Maximization (EM) algorithm for automatic parameter estimation and model selection (i.e., selecting the optimal number of mixture Gaussian components). (d) Subjective driven concept-oriented video skimming algorithm via semantic video classification

[1]  Lawrence Wai-Choong Wong,et al.  ANSES: Summarisation of News Video , 2003, CIVR.

[2]  Shih-Fu Chang,et al.  Determining computable scenes in films and their structures using audio-visual memory models , 2000, ACM Multimedia.

[3]  Jianping Fan,et al.  Multimodal Salient Objects: General Building Blocks of Semantic Video Concepts , 2004, CIVR.

[4]  Jianping Fan,et al.  Automatic image segmentation by integrating color-edge extraction and seeded region growing , 2001, IEEE Trans. Image Process..

[5]  Shih-Fu Chang.  Optimal Video Adaptation and Skimming Using a Utility-Based Framework , 2002 .

[6]  John R. Smith,et al.  Modal Keywords, Ontologies, and Reasoning for Video Understanding , 2003, CIVR.

[7]  William I. Grosky,et al.  Negotiating the semantic gap: from feature maps to semantic landscapes , 2001, Pattern Recognit..

[8]  Jianping Fan,et al.  Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing , 2004, IEEE Transactions on Image Processing.

[9]  Michael A. Smith,et al.  Video skimming and characterization through the combination of image and language understanding techniques , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Shin'ichi Satoh,et al.  Detection of important segments in cooking videos , 2001, Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL 2001).

[11]  Jianping Fan,et al.  Semantic video classification by integrating flexible mixture model with adaptive EM algorithm , 2003, MIR '03.

[12]  Alberto Del Bimbo,et al.  Model checking for detection of sport highlights , 2003, MIR '03.

[13]  Chong-Wah Ngo,et al.  Automatic video summarization by graph modeling , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[14]  Alan Hanjalic,et al.  Automated high-level movie segmentation for advanced video-retrieval systems , 1999, IEEE Trans. Circuits Syst. Video Technol..

[15]  Svetha Venkatesh,et al.  Towards automatic extraction of expressive elements from motion pictures: tempo , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[16]  Rainer Lienhart,et al.  Abstracting home video automatically , 1999, MULTIMEDIA '99.

[17]  Jianping Fan,et al.  Principal Video Shot: Linking Low-Level Perceptional Features to Semantic Video Events , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[18]  John R. Smith,et al.  Semantic Indexing of Multimedia Content Using Visual, Audio, and Text Cues , 2003, EURASIP J. Adv. Signal Process..

[19]  Svetha Venkatesh,et al.  Toward automatic extraction of expressive elements from motion pictures: tempo , 2002, IEEE Trans. Multim..

[20]  Theo Gevers,et al.  Robust segmentation and tracking of colored objects in video , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Shih-Fu Chang,et al.  Structure analysis of soccer video with hidden Markov models , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[22]  Shih-Fu Chang,et al.  Structure analysis of sports video using domain models , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[23]  Yihong Gong,et al.  Automatic parsing and indexing of news video , 1995, Multimedia Systems.