Examining Audience Retention in Educational Videos - Potential and Method

In-depth analysis of viewing behavior is an important factor informing educational video design. Research has largely addressed the support of such analyses for individual learners or has relied on aggregating user interactions with video players to derive potential points of interest. Only in recent years, with rising use of commercial video hosting platform, the data necessary to analyze viewing behavior on a larger scale and independently of active user interactions with learning content has become widely available. This article proposes to examine audience retention data to obtain in-depth information about the usage of educational videos and calculate metrics that can serve as a starting point for qualitatively examining the effects of didactic video design decisions. We propose a set of metrics and present a toolchain for extracting the necessary data from YouTube Analytics and calculating these metrics. The usefulness of these metrics is showcased on three different scenarios of use that are backed with real-world analytics data from flipped classroom courses. Our results show that the detailed audience retention data provided by video hosting platforms can provide insights that go beyond the current state-of-the-art in in-video learning analytics.

[1]  Georgios Evangelidis,et al.  Using metrics and cluster analysis for analyzing learner video viewing behaviours in educational videos , 2014, 2014 IEEE/ACS 11th International Conference on Computer Systems and Applications (AICCSA).

[2]  Robin H. Kay,et al.  Exploring the use of video podcasts in education: A comprehensive review of the literature , 2012, Comput. Hum. Behav..

[3]  John C. Strikwerda,et al.  Reversing the Lecture/Homework Paradigm Using eTEACH® Web‐based Streaming Video Software , 2002 .

[4]  Krzysztof Z. Gajos,et al.  Understanding in-video dropouts and interaction peaks inonline lecture videos , 2014, L@S.

[5]  Kleftodimos Alexandros,et al.  A Framework for Recording, Monitoring and Analyzing Learner Behavior while Watching and Interacting with Online Educational Videos , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[6]  Adrienne E. Williams,et al.  Lecture capture podcasts: differential student use and performance in a large introductory course , 2016 .

[7]  Michail N. Giannakos,et al.  Video-based learning ecosystem to support active learning: application to an introductory computer science course , 2016, Smart Learning Environments.

[8]  K. H. Vincent Lau,et al.  Using learning analytics to evaluate a video-based lecture series , 2018, Medical teacher.

[9]  Jonathan T. Copley,et al.  Audio and video podcasts of lectures for campus‐based students: production and evaluation of student use , 2007 .

[10]  Carlos Delgado Kloos,et al.  Precise Effectiveness Strategy for analyzing the effectiveness of students with educational resources and activities in MOOCs , 2015, Comput. Hum. Behav..

[11]  Georgios Evangelidis,et al.  A Framework for Recording, Monitoring and Analyzing Learner Behavior while Watching and Interacting with Online Educational Videos , 2013, ICALT.

[12]  Samir Chatterjee,et al.  A Design Science Research Methodology for Information Systems Research , 2008 .

[13]  Liz Jones,et al.  The use of lecture recordings in higher education: A review of institutional, student, and lecturer issues , 2015, Education and Information Technologies.

[14]  Chih-Ming Chen,et al.  Effects of Different Video Lecture Types on Sustained Attention, Emotion, Cognitive Load, and Learning Performance , 2015, IIAI-AAI.

[15]  Daniel Belanche,et al.  The effects of learner-generated videos for YouTube on learning outcomes and satisfaction , 2016, Comput. Educ..

[16]  Yannet Interian,et al.  Measuring Advertising Quality on Television , 2009, Journal of Advertising Research.

[17]  Georgios Evangelidis,et al.  Using open source technologies and open internet resources for building an interactive video based learning environment that supports learning analytics , 2016, Smart Learning Environments.

[18]  Deniz Gündüz,et al.  Audience retention rate aware coded video caching , 2017, 2017 IEEE International Conference on Communications Workshops (ICC Workshops).

[19]  Michail N. Giannakos,et al.  Making Sense of Video Analytics: Lessons Learned from Clickstream Interactions, Attitudes, and Learning Outcome in a Video-Assisted Course. , 2015 .

[20]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[21]  Qing Chen,et al.  PeakVizor: Visual Analytics of Peaks in Video Clickstreams from Massive Open Online Courses , 2016, IEEE Transactions on Visualization and Computer Graphics.

[22]  Tomoko Traphagan,et al.  Impact of class lecture webcasting on attendance and learning , 2010 .

[23]  Ruimin Shen,et al.  Analyzing Student Viewing Patterns in Lecture Videos , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[24]  C. Phillips,et al.  The use of flipped classrooms in higher education: A scoping review , 2015, Internet High. Educ..

[25]  Eric D. Manley,et al.  Video-based instruction for introductory computer programming , 2014 .

[26]  Krzysztof Z. Gajos,et al.  Data-driven interaction techniques for improving navigation of educational videos , 2014, UIST.