Exploring Online Learners' Interactive Dynamics by Visually Analyzing Their Time‐anchored Comments

MOOCs (Massive Open Online Courses) are increasingly prevalent as an online educational resource open to everyone and have attracted hundreds of thousands learners enrolling these online courses. At such scale, there is potentially rich information of learners' behaviors embedded in the interactions between learners and videos that may help instructors and content producers adjust the instructions and refine the online courses. However, the lack of tools to visualize information from interactive data, including messages left to the videos at particular timestamps as well as the temporal variations of learners' online participation and perceived experience, has prevented people from gaining more insights from video‐watching logs. In this paper, we focus on extracting and visualizing useful information from time‐anchored comments that learners left to specific time points of the videos when watching them. Timestamps as a kind of metadata of messages can be useful to recover the interactive dynamics of learners occurring around the videos. Therefore, we present a visualization system to analyze and categorize time‐anchored comments based on topics and content types. Our system integrates visualization methods of temporal text data, namely ToPIN and ThemeRiver, which can help people understand the quality and quantity of online learners' feedback and their states of learning. To evaluate the proposed system, we visualized time‐anchored commenting data from two online course videos, and conducted two user studies participated by course instructors and third‐party educational evaluators. The results validate the usefulness of the approach and show how the quantitative and qualitative visualizations can be used to gain interesting insights around learners' online learning behaviors.

[1]  Fangzhao Wu,et al.  OpinionFlow: Visual Analysis of Opinion Diffusion on Social Media , 2014, IEEE Transactions on Visualization and Computer Graphics.

[2]  Lucy T. Nowell,et al.  ThemeRiver: Visualizing Thematic Changes in Large Document Collections , 2002, IEEE Trans. Vis. Comput. Graph..

[3]  Kenneth Y. Goldberg,et al.  Opinion space: a scalable tool for browsing online comments , 2010, CHI.

[4]  Philip J. Guo,et al.  How video production affects student engagement: an empirical study of MOOC videos , 2014, L@S.

[5]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[6]  Qing Chen,et al.  VisMOOC: Visualizing video clickstream data from massive open online courses , 2015, 2014 IEEE Conference on Visual Analytics Science and Technology (VAST).

[7]  David McMenemy,et al.  A classification scheme for content analyses of YouTube video comments , 2013, J. Documentation.

[8]  Yoav Bergner,et al.  Who does what in a massive open online course? , 2014, Commun. ACM.

[9]  Yingcai Wu,et al.  EvoRiver: Visual Analysis of Topic Coopetition on Social Media , 2014, IEEE Transactions on Visualization and Computer Graphics.

[10]  David E. Pritchard,et al.  Studying Learning in the Worldwide Classroom Research into edX's First MOOC. , 2013 .

[11]  Xiaojuan Ma,et al.  VideoForest: interactive visual summarization of video streams based on danmu data , 2016, SIGGRAPH Asia Symposium on Visualization.

[12]  Krzysztof Z. Gajos,et al.  Data-driven interaction techniques for improving navigation of educational videos , 2014, UIST.

[13]  Xin Tong,et al.  TextFlow: Towards Better Understanding of Evolving Topics in Text , 2011, IEEE Transactions on Visualization and Computer Graphics.

[14]  Jian Zhao,et al.  Visual Analysis of MOOC Forums with iForum , 2017, IEEE Transactions on Visualization and Computer Graphics.

[15]  Giuseppe Carenini,et al.  ConVis: A Visual Text Analytic System for Exploring Blog Conversations , 2014, Comput. Graph. Forum.

[16]  Koichi Mori,et al.  Nokia internet pulse: a long term deployment and iteration of a twitter visualization , 2012, CHI EA '12.

[17]  Anna-Lan Huang,et al.  Similarity Measures for Text Document Clustering , 2008 .

[18]  Armando Fox,et al.  Monitoring MOOCs: which information sources do instructors value? , 2014, L@S.

[19]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[20]  Daniel A. Keim,et al.  ConToVi: Multi‐Party Conversation Exploration using Topic‐Space Views , 2016, Comput. Graph. Forum.

[21]  Krzysztof Z. Gajos,et al.  Understanding in-video dropouts and interaction peaks inonline lecture videos , 2014, L@S.

[22]  Ben Shneiderman,et al.  TopicFlow: Visualizing topic alignment of Twitter data over time , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[23]  Hao-Chuan Wang,et al.  Using Time-Anchored Peer Comments to Enhance Social Interaction in Online Educational Videos , 2015, CHI.

[24]  Qing Chen,et al.  PeakVizor: Visual Analytics of Peaks in Video Clickstreams from Massive Open Online Courses , 2016, IEEE Transactions on Visualization and Computer Graphics.

[25]  Martin Wattenberg,et al.  Stacked Graphs – Geometry & Aesthetics , 2008, IEEE Transactions on Visualization and Computer Graphics.

[26]  D. Kirkpatrick.  Techniques for evaluating training programs , 1979 .

[27]  Jean-Daniel Fekete,et al.  PolemicTweet: Video Annotation and Analysis through Tagged Tweets , 2013, INTERACT.

[28]  Simon Cross,et al.  Evaluation of the OLDS MOOC curriculum design course: participant perspectives, expectations and experiences , 2013 .

[29]  Ben Shneiderman,et al.  Monitoring Academic Conferences: Real-Time Visualization and Retrospective Analysis of Backchannel Conversations , 2012, 2012 International Conference on Social Informatics.

[30]  Martin Wattenberg,et al.  Conversation thumbnails for large-scale discussions , 2003, CHI Extended Abstracts.

[31]  M. Sheelagh T. Carpendale,et al.  SparkClouds: Visualizing Trends in Tag Clouds , 2010, IEEE Transactions on Visualization and Computer Graphics.

[32]  Vania Dimitrova,et al.  CourseVis: A graphical student monitoring tool for supporting instructors in web-based distance courses , 2007, Int. J. Hum. Comput. Stud..

[33]  Nancy Argüelles,et al.  Author ' s , 2008 .

[34]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[35]  Qing Chen,et al.  DropoutSeer: Visualizing learning patterns in Massive Open Online Courses for dropout reasoning and prediction , 2016, 2016 IEEE Conference on Visual Analytics Science and Technology (VAST).