Ontology-driven Annotation and Access of Educational Video Data in E-learning

In the past decade, we have witnessed unprecedented advances in multimedia technology. As a result, an unprecedented amount of multimedia data is being generated. Among the myriad types of multimedia data, presentation videos from lectures, conferences and seminars, and corporate trainings are of particular interest to this chapter. The need for specific solutions in this field comes from the popularity of e-learning systems. Recent years, there have been extensive efforts at both universities and colleges on developing e-learning systems to support distant learning. Based on Sloan Consortium survey (Allen & Seaman, 2008), over 3.9 million students were taking at least one online course during the fall 2007, which accounts for over 20% of all U.S. higher education students. Online enrolments continue to grow at rates far in excess of the total higher education student population, with the most recent data demonstrating no signs of slowing down. For example, the online enrolment growth rate for 2007 is 12.9%, while the growth rate for the overall higher education student population is only 1.2% for the same period (Allen & Seaman, 2008). In addition, there are e-learning systems for military, medical, and cooperate trainings (Smith, Ruocco, & Jansen, 1999; Fan, Luo, & Elmagarmid, 2004). For example, Microsoft supported 367 on-line training lectures with more than 9000 online viewers in the year of 1999 alone (He, Grudin, & Gupta, 2000). These e-learning systems enhance learning experiences and augment teachers' work in and out of traditional classrooms (Abowd, Brotherton, & Bhalodai, 1998; Flachsbart, Franklin, & Hammond, 2000). Working professionals as well embrace e-learning programs due to their convenience and flexibility (Kariya, 2003). However, due to unstructured and linear features of videos, the essential instructional content of most e-learning systems, the presentation videos, has not been fully exploited. People often feel difficulties in locating a specific piece of information in a presentation video. Sometimes they have to play back and forth several times to locate the right spot. To ensure effective exploitation of these video assets, efficient and flexible access mechanisms must be provided. Video annotation data play a critical role in video systems. The richer the annotation data are, the more flexible the video access becomes, and thus the more effective the video data can be utilized. We view video annotation as a two-step process: video segmentation, and 16

[1]  Raphaël Troncy,et al.  Designing and Using an Audio-Visual Description Core Ontology , 2004 .

[2]  Jianping Fan,et al.  Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing , 2004, IEEE Transactions on Image Processing.

[3]  Bob J. Wielinga,et al.  Ontology-Based Photo Annotation , 2001, IEEE Intell. Syst..

[4]  David Franklin,et al.  Improving human computer interaction in a classroom environment using computer vision , 2000, IUI '00.

[5]  Chong-Wah Ngo,et al.  Structuring lecture videos for distance learning applications , 2003, Fifth International Symposium on Multimedia Software Engineering, 2003. Proceedings..

[6]  Ingemar J. Cox,et al.  An optimized interaction strategy for Bayesian relevance feedback , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[7]  Jane Hunter,et al.  Adding Multimedia to the Semantic Web: Building an MPEG-7 ontology , 2001, SWWS.

[8]  Arjeh M. Cohen,et al.  Synchronized Multimedia Integration Language (SMIL) 2.0 , 1998 .

[9]  Marcel Worring,et al.  Building a visual ontology for video retrieval , 2005, MULTIMEDIA '05.

[10]  Gregory D. Abowd,et al.  Classroom 2000: a system for capturing and accessing multimedia classroom experiences , 1998, CHI Conference Summary.

[11]  Tetsuo Funada,et al.  Subtopic segmentation in lecture speech for the creation of lecture video contents , 2006, Systems and Computers in Japan.

[12]  Anoop Gupta,et al.  Automating lecture capture and broadcast: technology and videography , 2004, Multimedia Systems.

[13]  I. E. Allen,et al.  Staying the course: online education in the United States, 2008 , 2008 .

[14]  Jay F. Nunamaker,et al.  Automated Video Segmentation for Lecture Videos: A Linguistics-Based Approach , 2005, Int. J. Technol. Hum. Interact..

[15]  Dennis McLeod,et al.  Audio structuring and personalized retrieval using ontologies , 2000, Proceedings IEEE Advances in Digital Libraries 2000.

[16]  S. Kariya,et al.  Online education expands and evolves , 2003 .

[17]  Ingemar J. Cox,et al.  Target testing and the PicHunter Bayesian multimedia retrieval system , 1996, Proceedings of the Third Forum on Research and Technology Advances in Digital Libraries,.

[18]  Kunio Fukunaga,et al.  Blackboard segmentation using video image of lecture and its applications , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[19]  Ingemar J. Cox,et al.  Psychophysical studies of the performance of an image database retrieval system , 1998, Electronic Imaging.

[20]  Svetha Venkatesh,et al.  High level segmentation of instructional videos based on content density , 2002, MULTIMEDIA '02.

[21]  Eero Hyvönen,et al.  Ontology-Based Image Retrieval , 2003, WWW.

[22]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[23]  N. F. Noy,et al.  Ontology Development 101: A Guide to Creating Your First Ontology , 2001 .

[24]  Anoop Gupta,et al.  Designing presentations for on-demand viewing , 2000, CSCW '00.

[25]  Tom Minka,et al.  Interactive learning with a "Society of Models" , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Marti A. Hearst Text Tiling: Segmenting Text into Multi-paragraph Subtopic Passages , 1997, CL.

[27]  Tom Minka,et al.  Vision texture for annotation , 1995, Multimedia Systems.

[28]  Gregory D. Abowd,et al.  Lessons learned from eClass: Assessing automated capture and access in the classroom , 2004, TCHI.

[29]  Alexander G. Hauptmann,et al.  Towards a Large Scale Concept Ontology for Broadcast Video , 2004, CIVR.

[30]  Bernard J. Jansen,et al.  Digital video in education , 1999, SIGCSE '99.

[31]  Zhang Hai-ling Semantic Integration and Retrieval of Multimedia Metadata , 2007 .

[32]  John R. Kender,et al.  Analysis and interface for instructional video , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[33]  David Wheeler,et al.  Building Customized Data Pipelines Using the Entrez Programming Utilities (eUtils) , 2004 .

[34]  Diane Harley,et al.  BIBS: A Lecture Webcasting System , 2001 .

[35]  Brian Christopher Smith,et al.  Passive capture and structuring of lectures , 1999, MULTIMEDIA '99.

[36]  Yasuo Ariki,et al.  Topic segmentation and retrieval system for lecture videos based on spontaneous speech recognition , 2003, INTERSPEECH.

[37]  Vasant Honavar,et al.  Integration of Domain-Specific and Domain-Independent Ontologies for Colonoscopy Video Database Annotation , 2004, IKE.

[38]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[39]  C. Tsinaraki,et al.  Interoperability Support for Ontology-Based Video Retrieval Applications , 2004, CIVR.

[40]  John R. Kender,et al.  Rule-based semantic summarization of instructional videos , 2002, Proceedings. International Conference on Image Processing.