Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement

The learning analytics dashboard (LAD) is a newly developed learning support tool for virtual classrooms that is believed to allow students to review their online learning behavior patterns intuitively through the provision of visual information. The purpose of this study was to empirically validate the effects of LAD. An experimental study was conducted with a dashboard treatment group and a control group. The researchers developed a LAD and evaluated its effectiveness on the sample of 151 college students at a private university located in Korea, who were taking the online course titled “Management Statistics” in the first semester of 2014. The following results were obtained. First, the students who received dashboard treatment presented higher final score than those who did not. Second, the dashboard usage frequency, as measured by the number of times the dashboard was opened, did not have a significant impact on learning achievement. However, a slightly positive correlation between satisfaction with LAD and learning achievement was observed. Further analysis indicated that learners who used the dashboard only a few times showed relatively high satisfaction with LAD. On the other hand, high academic achievers who opened LAD relatively frequently showed lower satisfaction with dashboard. The results guide that LAD should be revised in a way to motivate learners consistently and support learners who have different academic achievement levels. The study discusses the further research tasks in terms of LAD development as an effective and personalized feedback tool to improve learners’ academic achievement.

[1]  Yeonjeong Park,et al.  Tracking Students' Eye-Movements on Visual Dashboard Presenting Their Online Learning Behavior Patterns , 2014, ICSLE.

[2]  S. Kuhar,et al.  Taking Advantage of Education Data: Advanced Data Analysis and Reporting in Virtual Learning Environments , 2011 .

[3]  Meehyun Yoon,et al.  Analyzing the log patterns of adult learners in LMS using learning analytics , 2014, LAK.

[4]  Erik Duval,et al.  Learning Analytics Dashboard Applications , 2013 .

[5]  E. Duval.  Attention please!: learning analytics for visualization and recommendation , 2011, LAK.

[6]  George Siemens,et al.  Learning Analytics , 2013 .

[7]  B. Zimmerman,et al.  Self-regulated learning and academic achievement : theoretical perspectives , 2001 .

[8]  Il-Hyun Jo,et al.  Educational technology approach toward learning analytics: relationship between student online behavior and learning performance in higher education , 2014, LAK '14.

[9]  Erik Duval,et al.  Goal-oriented visualizations of activity tracking: a case study with engineering students , 2012, LAK.

[10]  Marek Hatala,et al.  A qualitative evaluation of evolution of a learning analytics tool , 2012, Comput. Educ..

[11]  Claus Zinn,et al.  How did the e-learning session go? The Student Inspector , 2007, AIED.

[12]  Erik Duval,et al.  The student activity meter for awareness and self-reflection , 2012, CHI Extended Abstracts.

[13]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[14]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[15]  Ann P. Daunic,et al.  Treatment Fidelity in Applied Educational Research: Expanding the Adoption and Application of Measures to Ensure Evidence-Based Practice , 2007 .

[16]  Vania Dimitrova,et al.  Using Student and Group Models to Support Teachers in Web-Based Distance Education , 2005, User Modeling.

[17]  Tanya Elias,et al.  Learning Analytics: Definitions, Processes and Potential , 2011 .

[18]  Jo IlHyun,et al.  Design and Application of Visual Dashboard Based on Learning Analytics , 2014 .

[19]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[20]  Alyssa Friend Wise,et al.  Designing pedagogical interventions to support student use of learning analytics , 2014, LAK.

[21]  Ulrik Schroeder,et al.  A reference model for learning analytics , 2012 .

[22]  Yeonjeong Park,et al.  Educational Dashboards for Smart Learning: Review of Case Studies , 2014, ICSLE.

[23]  George Siemens,et al.  Learning analytics and educational data mining: towards communication and collaboration , 2012, LAK.

[24]  Paul S. Steif,et al.  Web-based Statics Course with Learning Dashboard for Instructors , 2012 .

[25]  Il-Hyun Jo,et al.  Impact of Learner’s Time Management Strategies on Achievement in an e-learning Environment: A Learning Analytics Approach , 2013 .

[26]  Il-Hyun Jo,et al.  Usability Study of Middle School English Digital Textbook: A Stimulated Recall Approach , 2013 .

[27]  Gautam Biswas,et al.  Shallow strategy development in a teachable agent environment designed to support self-regulated learning , 2013, Comput. Educ..

[28]  Jeff Grann,et al.  Competency map: visualizing student learning to promote student success , 2014, LAK.

[29]  Yeonjeong Park,et al.  Development of the Learning Analytics Dashboard to Support Students' Learning Performance , 2015, J. Univers. Comput. Sci..

[30]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[31]  Matthew D. Pistilli,et al.  In practice: Purdue Signals: Mining real‐time academic data to enhance student success , 2010 .

[32]  G. D. Chen,et al.  Ubiquitous learning website: Scaffold learners by mobile devices with information-aware techniques , 2008, Comput. Educ..

[33]  Alejandra Martínez-Monés,et al.  From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning , 2005, Int. J. Artif. Intell. Educ..