Indexed Captioned Searchable Videos: A Learning Companion for STEM Coursework

Videos of classroom lectures have proven to be a popular and versatile learning resource. A key shortcoming of the lecture video format is accessing the content of interest hidden in a video. This work meets this challenge with an advanced video framework featuring topical indexing, search, and captioning (ICS videos). Standard optical character recognition (OCR) technology was enhanced with image transformations for extraction of text from video frames to support indexing and search. The images and text on video frames is analyzed to divide lecture videos into topical segments. The ICS video player integrates indexing, search, and captioning in video playback providing instant access to the content of interest. This video framework has been used by more than 70 courses in a variety of STEM disciplines and assessed by more than 4000 students. Results presented from the surveys demonstrate the value of the videos as a learning resource and the role played by videos in a students learning process. Survey results also establish the value of indexing and search features in a video platform for education. This paper reports on the development and evaluation of ICS videos framework and over 5 years of usage experience in several STEM courses.

[1]  Arijit Biswas,et al.  MMToC: A Multimodal Method for Table of Content Creation in Educational Videos , 2015, ACM Multimedia.

[2]  Jeffrey R. Young,et al.  Inside the Coursera Contract: How an Upstart Company Might Profit from Free Courses. , 2012 .

[3]  Craig Ross,et al.  The timing of online lecture slide availability and its effect on attendance, participation, and exam performance , 2009, Comput. Educ..

[4]  Nikolai Joukov,et al.  Lectern II: a multimedia lecture capturing and editing system , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[5]  T. Burne,et al.  Digital lecture recording: a cautionary tale. , 2013, Nurse education in practice.

[6]  John R. Kender,et al.  Semantic keyword extraction via adaptive text binarization of unstructured unsourced video , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[7]  Jun Ye,et al.  Cosine similarity measures for intuitionistic fuzzy sets and their applications , 2011, Math. Comput. Model..

[8]  Khe Foon Hew,et al.  Use of audio podcast in K-12 and higher education: a review of research topics and methodologies , 2009 .

[9]  Gregory D. Abowd,et al.  Classroom 2000: An Experiment with the Instrumentation of a Living Educational Environment , 1999, IBM Syst. J..

[10]  Anna-Lan Huang,et al.  Similarity Measures for Text Document Clustering , 2008 .

[11]  Flora McMartin,et al.  Costs, Culture, and Complexity: An Analysis of Technology Enhancements in a Large Lecture Course at UC Berkeley , 2003 .

[12]  Jignesh M. Patel,et al.  Estimating the selectivity of tf-idf based cosine similarity predicates , 2007, SGMD.

[13]  C. Chandrasekar,et al.  A Comparison of various Edge Detection Techniques used in Image Processing , 2012 .

[14]  Fouad A. Tobagi.  Distance Learning with Digital Video , 1995, IEEE Multim..

[15]  Hasan Al-Nashash,et al.  Lecture Capture in Engineering Classes: Bridging Gaps and Enhancing Learning , 2013, J. Educ. Technol. Soc..

[16]  Arding Hsu,et al.  Image processing on compressed data for large video databases , 1993, MULTIMEDIA '93.

[17]  Lecia Jane Barker,et al.  A crowdsourcing caption editor for educational videos , 2014, 2014 IEEE Frontiers in Education Conference (FIE) Proceedings.

[18]  Hamad Odhabi,et al.  Video recording lectures: Student and professor perspectives , 2011, Br. J. Educ. Technol..

[19]  David Hung-Chang Du,et al.  Video-Based Hypermedia for Education-on-Demand , 1998, IEEE Multim..

[20]  Rakesh M. Verma,et al.  Topic based segmentation of classroom videos , 2015, 2015 IEEE Frontiers in Education Conference (FIE).

[21]  Ian N. Toppin.  Video lecture capture (VLC) system: A comparison of student versus faculty perceptions , 2011, Education and Information Technologies.

[22]  Yoshinobu Tonomura,et al.  Projection Detecting Filter for Video Cut Detection , 1993, ACM Multimedia.

[23]  Gregory D. Abowd,et al.  Lessons learned from eClass: Assessing automated capture and access in the classroom , 2004, TCHI.

[24]  Thomas D. C. Little,et al.  A Survey of Technologies for Parsing and Indexing Digital Video1 , 1996, J. Vis. Commun. Image Represent..

[25]  Christian Huemer,et al.  Replacing traditional classroom lectures with lecture videos: an experience report , 2012, EduSymp '12.

[26]  Marco Ronchetti,et al.  Video-lectures in a traditional mathematics course on iTunes U: Students' feedback , 2011, 2011 14th International Conference on Interactive Collaborative Learning.

[27]  Surendar Chandra,et al.  Experiences in Personal Lecture Video Capture , 2011, IEEE Transactions on Learning Technologies.

[28]  Jonathan T. Copley,et al.  Audio and video podcasts of lectures for campus‐based students: production and evaluation of student use , 2007 .

[29]  Raj Acharya,et al.  Connected component labeling with linear octree , 1991, Pattern Recognit..

[30]  Mark Grabe,et al.  Evaluating the advantages and disadvantages of providing lecture notes: The role of internet technology as a delivery system and research tool , 2005, Internet High. Educ..

[31]  Akio Nagasaka,et al.  Automatic Video Indexing and Full-Video Search for Object Appearances , 1991, VDB.

[32]  John Adcock,et al.  TalkMiner: a lecture webcast search engine , 2010, ACM Multimedia.

[33]  Ramesh C. Jain,et al.  Production model based digital video segmentation , 1995, Multimedia Tools and Applications.

[34]  Andy Cockburn,et al.  Flexible Delivery Damaging to Learning? Lessons from the Canterbury Digital Lectures Project. , 2001 .

[35]  Michael Bianchi.  Automatic video production of lectures using an intelligent and aware environment , 2004, MUM '04.

[36]  Susanto Rahardja,et al.  Video retrieval - evolution of video segmentation, indexing and search , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[37]  Jay F. Nunamaker,et al.  Segmentation of lecture videos based on text: a method combining multiple linguistic features , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[38]  W Sterry,et al.  Online lectures for students in dermatology: A replacement for traditional teaching or a valuable addition? , 2009, Journal of the European Academy of Dermatology and Venereology : JEADV.

[39]  Lecia Jane Barker,et al.  Development and evaluation of indexed captioned searchable videos for STEM coursework , 2012, SIGCSE '12.

[40]  Jaspal Subhlok,et al.  Indexing and keyword search to ease navigation in lecture videos , 2011, 2011 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[41]  Mark Grabe,et al.  Voluntary use of online lecture notes: correlates of note use and note use as an alternative to class attendance , 2005, Comput. Educ..

[42]  Jenny A. Van Amburgh,et al.  Online lecture delivery paired with in class problem-based learning … Does it enhance student learning? , 2011 .

[43]  Dirk Schönfuß,et al.  CONTENTUS—technologies for next generation multimedia libraries , 2011, Multimedia Tools and Applications.

[44]  Chris Evans,et al.  The effectiveness of m-learning in the form of podcast revision lectures in higher education , 2008, Comput. Educ..

[45]  Tomoko Traphagan,et al.  Impact of class lecture webcasting on attendance and learning , 2010 .

[46]  Christopher Lynnly Hovey,et al.  Student perceptions of indexed, searchable videos of faculty lectures , 2014, 2014 IEEE Frontiers in Education Conference (FIE) Proceedings.

[47]  G. T. Shrivakshan,et al.  A Comparison of various Edge Detection Techniques used in Image Processing , 2012 .

[48]  Surendar Chandra,et al.  Lecture video capture for the masses , 2007, ITiCSE.

[49]  Shin'ichi Satoh,et al.  Key Image Extraction from a News Video Archive for Visualizing Its Semantic Structure , 2004, PCM.

[50]  W. Richards Adrion,et al.  Student reactions to classroom lecture capture , 2012, ITiCSE '12.

[51]  Christoph Meinel,et al.  Lecture Video Browsing Using Multimodal Information Resources , 2013, ICWL.

[52]  Blanche W. O'Bannon,et al.  Using podcasts to replace lecture: Effects on student achievement , 2011, Comput. Educ..

[53]  Wolfgang Effelsberg,et al.  Automatic text segmentation and text recognition for video indexing , 2000, Multimedia Systems.