Lecture Video Browsing Using Multimodal Information Resources

In the last decade e-lecturing has become more and more popular. The amount of lecture video data on the World Wide Web WWW is growing rapidly. Therefore, a more efficient method for video retrieval in WWW or within large lecture video archives is urgently needed. This paper presents an approach for automated video indexing and video search in large lecture video archives. First of all, we apply automatic video segmentation and key-frame detection to offer a visual guideline for the video content navigation. Subsequently, we extract textual metadata by applying video Optical Character Recognition OCR technology on key-frames and by performing Automatic Speech Recognition ASR on lecture audio tracks. The OCR and ASR transcript as well as detected slide text line types are adopted for keyword extraction, by which both video- and segment-level keywords are extracted respectively. Furthermore, we developed a content-based video search function and conducted a user study for evaluating the performance and the effectiveness of proposed indexing methods in our lecture video archive.

[1]  Gary Geunbae Lee,et al.  A Korean Spoken Document Retrieval System for Lecture Search , 2008 .

[2]  Dirk Schönfuß,et al.  CONTENTUS—technologies for next generation multimedia libraries , 2011, Multimedia Tools and Applications.

[3]  Harald Sack,et al.  Towards exploratory video search using linked data , 2009, 2009 11th IEEE International Symposium on Multimedia.

[4]  Harald Sack,et al.  Lecture Video Indexing and Analysis Using Video OCR Technology , 2011, 2011 Seventh International Conference on Signal Image Technology & Internet-Based Systems.

[5]  Mauro Cettolo,et al.  Language modeling and transcription of the TED corpus lectures , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[6]  Christoph Meinel,et al.  An Automated Analysis and Indexing Framework for Lecture Video Portal , 2012, ICWL.

[7]  Chong-Wah Ngo,et al.  Structuring low-quality videotaped lectures for cross-reference browsing by video text analysis , 2008, Pattern Recognit..

[8]  John R. Kender,et al.  Augmented segmentation and visualization for presentation videos , 2005, MULTIMEDIA '05.

[9]  James R. Glass,et al.  Analysis and Processing of Lecture Audio Data: Preliminary Investigations , 2004, Proceedings of the Workshop on Interdisciplinary Approaches to Speech Indexing and Retrieval at HLT-NAACL 2004 - SpeechIR '04.

[10]  Jens Lehmann,et al.  DBpedia: A Nucleus for a Web of Open Data , 2007, ISWC/ASWC.

[11]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[12]  Ralf Klamma,et al.  Advances in Web-Based Learning - ICWL 2012 , 2012, Lecture Notes in Computer Science.

[13]  John Adcock,et al.  TalkMiner: a lecture webcast search engine , 2010, ACM Multimedia.

[14]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.