A natural language approach to content-based video indexing and retrieval for interactive e-learning

As a powerful and expressive nontextual media that can capture and present information, instructional videos are extensively used in e-learning (Web-based distance learning). Since each video may cover many subjects, it is critical for an e-learning environment to have content-based video searching capabilities to meet diverse individual learning needs. In this paper, we present an interactive multimedia-based e-learning environment that enables users to interact with it to obtain knowledge in the form of logically segmented video clips. We propose a natural language approach to content-based video indexing and retrieval to identify appropriate video clips that can address users' needs. The method integrates natural language processing, named entity extraction, frame-based indexing, and information retrieval techniques to explore knowledge-on-demand in a video-based interactive e-learning environment. A preliminary evaluation shows that precision and recall of this approach are better than those of the traditional keyword based approach.

[1]  Arnon Amir,et al.  Towards automatic real time preparation of on-line video proceedings for conference talks and presentations , 2001, Proceedings of the 34th Annual Hawaii International Conference on System Sciences.

[2]  Shih-Fu Chang,et al.  Video object model and segmentation for content-based video indexing , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[3]  Mubarak Shah,et al.  A framework for segmentation of interview videos , 2000 .

[4]  Stephen W. Smoliar,et al.  Content based video indexing and retrieval , 1994, IEEE MultiMedia.

[5]  Lawrence Birnbaum,et al.  War stories: harnessing organizational memories to support task performance , 2000, INTL.

[6]  Ellen M. Voorhees,et al.  The TREC-8 Question Answering Track Evaluation , 2000, TREC.

[7]  Howard D. Wactlar,et al.  Informedia - Search and Summarization in the Video Medium , 2000 .

[8]  H. L. Pugh,et al.  The Use of Videoteletraining to Deliver Hands-on Training: Concept Test and Evaluation , 1992 .

[9]  Sang Uk Lee,et al.  Efficient video indexing scheme for content-based retrieval , 1999, IEEE Trans. Circuits Syst. Video Technol..

[10]  Andreas Girgensohn,et al.  A genetic algorithm for video segmentation and summarization , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[11]  Mahbubur Rahman Syed.  Diminishing the distance in distance education , 2001 .

[12]  Kristian J. Hammond,et al.  Question Answering from Frequently Asked Question Files: Experiences with the FAQ FINDER System , 1997, AI Mag..

[13]  Ahmed K. Elmagarmid,et al.  WVTDB - A Semantic Content-Based Video Database System on the World Wide Web , 1998, IEEE Trans. Knowl. Data Eng..

[14]  Ramesh C. Jain,et al.  Video Data Management Systems: Metadata and Architecture , 1998, Multimedia Data Management.

[15]  Walter Anthony Cook.  Case Grammar Theory , 1979 .

[16]  Gerard Salton,et al.  The SMART Retrieval System , 1971 .

[17]  E. Bailey,et al.  Teaching via the Internet. , 1994 .

[18]  Nevenka Dimitrova,et al.  Video keyframe extraction and filtering: a keyframe is not a keyframe to everyone , 1997, CIKM '97.

[19]  John M. Gauch,et al.  The vision digital video library , 1997, Inf. Process. Manag..

[20]  A. Murat Tekalp,et al.  Object-based video indexing for virtual-studio productions , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Panos Kalnis,et al.  Content-based retrieval using heuristic search , 1999, SIGIR '99.

[22]  Minoru Mori,et al.  Telop-on-demand: video structuring and retrieval based on text recognition , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[23]  Wei Li,et al.  Information Extraction Supported Question Answering , 1999, TREC.

[24]  Rainer Lienhart,et al.  Automatic text recognition for video indexing , 1997, MULTIMEDIA '96.

[25]  Kenneth C. Litkowski,et al.  Syntactic Clues and Lexical Resources in Question-Answering , 2000, TREC.

[26]  Yihong Gong,et al.  Lessons Learned from Building a Terabyte Digital Video Library , 1999, Computer.

[27]  Wenjun Zeng,et al.  Integrated image and speech analysis for content-based video indexing , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[28]  Scott Stevens,et al.  Synthetic interviews: the art of creating a “dyad” between humans and machine-based characters , 1998, MULTIMEDIA '98.

[29]  Karen Spärck Jones,et al.  Open-vocabulary speech indexing for voice and video mail retrieval , 1997, MULTIMEDIA '96.

[30]  Ioannis Pitas,et al.  Content-based video parsing and indexing based on audio-visual interaction , 2001, IEEE Trans. Circuits Syst. Video Technol..

[31]  Stanley Boykin,et al.  Machine learning of event segmentation for news on demand , 2000, CACM.

[32]  Richard M. Schwartz,et al.  An Algorithm that Learns What's in a Name , 1999, Machine Learning.

[33]  Robin Burke,et al.  Supporting learning through active retrieval of video stories , 1995 .

[34]  Linda Carswell.  Teaching via the Internet: the impact of the Internet as a communication medium on distance learning introductory computing students , 1997, ITiCSE '97.

[35]  Douglas Keislar,et al.  Content-Based Classification, Search, and Retrieval of Audio , 1996, IEEE Multim..

[36]  Rangasami L. Kashyap,et al.  Models for motion-based video indexing and retrieval , 2000, IEEE Trans. Image Process..

[37]  Sheelagh Carville,et al.  'It's A Bit Like Star Trek': The Effectiveness of Video Conferencing , 2000 .

[38]  Ralph Grishman,et al.  The NYU System for MUC-6 or Where’s the Syntax? , 1995, MUC.

[39]  Robin D. Burke,et al.  Conceptual indexing and active retrieval of video for interactive learning environments , 1996, Knowl. Based Syst..

[40]  Edoardo Ardizzone,et al.  Automatic Video Database Indexing and Retrieval , 2004, Multimedia Tools and Applications.

[41]  Howard D. Wactlar,et al.  Complementary video and audio analysis for broadcast news archives , 2000, CACM.

[42]  Gregory D. Abowd,et al.  Supporting educational activities through dynamic web interfaces , 2001, Interact. Comput..

[43]  Yong Man Ro,et al.  Fast content-based MPEG video indexing using object motion , 1999, Proceedings of IEEE. IEEE Region 10 Conference. TENCON 99. 'Multimedia Technology for Asia-Pacific Information Infrastructure' (Cat. No.99CH37030).