Open Learning Analytics: A Systematic Literature Review and Future Perspectives

Open learning analytics (OLA) is an emerging research area that aims at improving learning efficiency and effectiveness in lifelong learning environments. OLA employs multiple methods to draw value from a wide range of educational data coming from various learning environments and contexts in order to gain insight into the learning processes of different stakeholders. As the research field is still relatively young, only a few technical platforms are available, and a common understanding of requirements is lacking. This paper provides a systematic literature review of tools available in the learning analytics literature from 2011 to 2019 with an eye on their support for openness. One hundred thirty-seven tools from nine academic databases are collected to form the base for this review. The analysis of selected tools is performed based on four dimensions, namely, “Data, Environments, Context (What?),” “Stakeholders (Who?),” “Objectives (Why?),” and “Methods (How?).” Moreover, five well-known OLA frameworks available in the community are systematically compared. The review concludes by eliciting the main requirements for an effective OLA platform and by identifying key challenges and future lines of work in this emerging field.

[1]  Andrii Vozniuk,et al.  Towards Portable Learning Analytics Dashboards , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[2]  Philippe J. Giabbanelli,et al.  Learning Analytics to Support Teachers’ Assessment of Problem Solving: A Novel Application for Machine Learning and Graph Algorithms , 2019, Utilizing Learning Analytics to Support Study Success.

[3]  Tetsuro Kakeshita,et al.  Analysis Method of Student Achievement Level Utilizing Web-Based Programming Education Support Tool Pgtacer , 2016, 2016 5th IIAI International Congress on Advanced Applied Informatics (IIAI-AAI).

[4]  Thrasyvoulos Tsiatsos,et al.  Evaluation of a Blog-Based Learning Analytics Tool: A Case Study Focusing on Teachers , 2017, IMCL.

[5]  George Siemens,et al.  Learning analytics and educational data mining: towards communication and collaboration , 2012, LAK.

[6]  Nitesh V. Chawla,et al.  Integrated Closed-loop Learning Analytics Scheme in a First Year Experience Course , 2019, LAK.

[7]  Davinia Hernández Leo,et al.  Supporting awareness in communities of learning design practice , 2018, Comput. Hum. Behav..

[8]  M. Schaaf.  Electronic Portfolios Enhanced with Learning Analytics at the Workplace , 2018 .

[9]  Indika Perera,et al.  Enhancing competencies of less-able students to achieve learning outcomes: Learner aware tool support through Business intelligence , 2016, 2016 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE).

[10]  Hiroaki Ogata,et al.  Connecting Dots for Ubiquitous Learning Analytics , 2015, ICHL.

[11]  Iván Martínez-Ortiz,et al.  Integrating Learning Analytics into a Game Authoring Tool , 2017, ICWL.

[12]  Thomas Lotze,et al.  Evolving a learning analytics platform , 2011, LAK.

[13]  Thanasis Daradoumis,et al.  Open Proctor: An Academic Integrity Tool for the Open Learning Environment , 2017, INCoS.

[14]  Quang Vinh Nguyen,et al.  IntelliViz- A Tool for Visualizing Social Networks with Hashtags , 2014, ISVC.

[15]  Skw Chu,et al.  A Mobile App Platform for Discovering Learning Profiles and Analytics , 2016 .

[16]  Ulrik Schroeder,et al.  A reference model for learning analytics , 2012 .

[17]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[18]  Vincent Aleven,et al.  The classroom as a dashboard: co-designing wearable cognitive augmentation for K-12 teachers , 2018, LAK.

[19]  Denis Gillet,et al.  An Activity Tracking Infrastructure for Embedded Open Educational Labs Supporting the Needs of Lab Owners and Students , 2017 .

[20]  Iván Martínez-Ortiz,et al.  Systematizing game learning analytics for serious games , 2017, 2017 IEEE Global Engineering Education Conference (EDUCON).

[21]  Mandy Lupton,et al.  Learning analytics beyond the LMS: the connected learning analytics toolkit , 2015, LAK.

[22]  Zdenek Zdráhal,et al.  Learning Analytics Dashboard Analysing First-Year Engineering Students , 2018, EC-TEL.

[23]  Theocharis Kyriacou,et al.  Student Centred Design of a Learning Analytics System , 2019, LAK.

[24]  Joel S. Mtebe,et al.  Using Learning Analytics to Predict Students’ Performance in Moodle Learning Management System: A Case of Mbeya University of Science and Technology , 2017, Electron. J. Inf. Syst. Dev. Ctries..

[25]  Katrien Verbert,et al.  Learning Analytics Dashboards to Support Adviser-Student Dialogue , 2018, IEEE Transactions on Learning Technologies.

[26]  David Griol,et al.  Increasing the Role of Data Analytics in m-Learning Conversational Applications , 2018 .

[27]  Eitel J. M. Lauría,et al.  Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative , 2014, J. Learn. Anal..

[28]  Susan Bull,et al.  A Novel Web-Based Approach for Visualization and Inspection of Reading Difficulties on University Students , 2017, IEEE Transactions on Learning Technologies.

[29]  Gonzalo Farias,et al.  Automated Assessment and Monitoring Support for Competency-Based Courses , 2019, IEEE Access.

[30]  Ulrik Schroeder,et al.  A Modular and Extensible Framework for Open Learning Analytics , 2018, J. Learn. Anal..

[31]  Nickolas J. G. Falkner,et al.  Using Learning Analytics to Visualise Computer Science Teamwork , 2015, ITiCSE.

[32]  Ali Daud,et al.  An integrated framework for course adapted student learning analytics dashboard , 2019, Comput. Hum. Behav..

[33]  Hanan Ayad,et al.  Improving student success using predictive models and data visualisations , 2012 .

[34]  Ramón Fabregat,et al.  A Software Suite for Efficient Use of the European Qualifications Framework in Online and Blended Courses , 2013, IEEE Transactions on Learning Technologies.

[35]  Ulrik Schroeder,et al.  Toward an Open Learning Analytics Ecosystem , 2017 .

[36]  Boban Vesin,et al.  Cross-Platform Analytics: A step towards Personalization and Adaptation in Education , 2019, LAK.

[37]  Ruth Cobos Pérez,et al.  A Learning Analytics Tool for Predictive Modeling of Dropout and Certificate Acquisition on MOOCs for Professional Learning , 2018, 2018 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM).

[38]  Vive Kumar,et al.  Dynamic Learning Path Prediction — A Learning Analytics Solution , 2016, 2016 IEEE Eighth International Conference on Technology for Education (T4E).

[39]  Dragan Gasevic,et al.  Recipe for success: lessons learnt from using xAPI within the connected learning analytics toolkit , 2016, LAK.

[40]  Atsushi Shimada,et al.  Integrating Multimodal Learning Analytics and Inclusive Learning Support Systems for People of All Ages , 2019, HCI.

[41]  Wolfgang Müller,et al.  Automatic Inspection of E-Portfolios for Improving Formative and Summative Assessment , 2016, SETE@ICWL.

[42]  Ulrik Schroeder,et al.  The Goal - Question - Indicator Approach for Personalized Learning Analytics - , 2017, CSEDU.

[43]  Rémi Venant,et al.  Evaluation of Learner Performance During Practical Activities: An Experimentation in Computer Education , 2016, 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT).

[44]  Vincenza Carchiolo,et al.  Monitoring students activities in CS courses , 2016, 2016 15th RoEduNet Conference: Networking in Education and Research.

[45]  Angélica de Antonio Jiménez,et al.  Visualizing a collective student model for procedural training environments , 2018, Multimedia Tools and Applications.

[46]  Sabine Graf,et al.  AAT: a tool for accessing and analysing students' behaviour data in learning systems , 2011, LAK.

[47]  Katrien Verbert,et al.  Dashboard for Actionable Feedback on Learning Skills: Scalability and Usefulness , 2017, HCI.

[48]  Stelios Xinogalos,et al.  Integrating Learning Analytics in an Educational MMORPG for Computer Programming , 2014, 2014 IEEE 14th International Conference on Advanced Learning Technologies.

[49]  Xavier Ochoa,et al.  The RAP system: automatic feedback of oral presentation skills using multimodal analysis and low-cost sensors , 2018, LAK.

[50]  Pierpaolo Di Bitonto,et al.  Digitally enhanced assessment in virtual learning environments , 2015, J. Vis. Lang. Comput..

[51]  Jaime D. L. Caro,et al.  Use of analytics to improve student behavior and performance in an online course implementation , 2015, 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA).

[52]  Anne Tervakari,et al.  Content analysis and visualizations: Tools for a social media-enhanced learning environment , 2014, 2014 International Conference on Interactive Collaborative Learning (ICL).

[53]  Danielle S. McNamara,et al.  ReaderBench: An Integrated Tool Supporting both Individual and Collaborative Learning , 2015 .

[54]  José A. Ruipérez-Valiente,et al.  Scaling to Massiveness With ANALYSE: A Learning Analytics Tool for Open edX , 2017, IEEE Transactions on Human-Machine Systems.

[55]  Alfred Essa,et al.  A Scalable Learning Analytics Platform for Automated Writing Feedback , 2016, EDM.

[56]  Liang-Chih Yu,et al.  Development of daily activity and lifestyle data visualization tool for the college student learning analytics , 2016, EDM 2016.

[57]  Christoph Meinel,et al.  Utilizing Web Analytics in the Context of Learning Analytics for Large-Scale Online Learning , 2019, 2019 IEEE Global Engineering Education Conference (EDUCON).

[58]  Nora M. Villanueva,et al.  SIMPLIFY ITS: An intelligent tutoring system based on cognitive diagnosis models and spaced learning , 2018, IEEE Global Engineering Education Conference.

[59]  Mohamed Amine Chatti,et al.  The PERLA Framework: Blending Personalization and Learning Analytics , 2019 .

[60]  Vive Kumar,et al.  Learning Analytics Solution for Reducing Learners' Course Failure Rate , 2015, 2015 IEEE Seventh International Conference on Technology for Education (T4E).

[61]  Bert Bredeweg,et al.  Learning Analytics Pilot with Coach2 - Searching for Effective Mirroring , 2016, EC-TEL.

[62]  Jordi Conesa,et al.  ICT-FLAG: a web-based e-assessment platform featuring learning analytics and gamification , 2017, Int. J. Web Inf. Syst..

[63]  Neil Peirce,et al.  Empowering L&D Managers through Customisation of Inline Learning Analytics , 2014, HCI.

[64]  Alyssa Friend Wise,et al.  Learning Analytics for Online Discussions: Embedded and Extracted Approaches , 2014, J. Learn. Anal..

[65]  Miroslav Minovic,et al.  Real-time learning analytics in educational games , 2013, TEEM '13.

[66]  Ulrik Schroeder,et al.  A rule-based indicator definition tool for personalized learning analytics , 2016, LAK.

[67]  Davide Taibi,et al.  A semantic approach to mobile learning analytics , 2013, TEEM '13.

[68]  Dragan Gasevic,et al.  Open Learning Analytics: an integrated modularized platform , 2011 .

[69]  Isaac L. Chuang,et al.  Google BigQuery for Education: Framework for Parsing and Analyzing edX MOOC Data , 2017, L@S.

[70]  Emanuele Ruffaldi,et al.  A Learning Analytics Framework for Practice-Based Learning , 2015, CSCL 2015.

[71]  Juan Manuel Dodero,et al.  Learning Analytics in Mobile Applications Based on Multimodal Interaction , 2018 .

[72]  Hendrik Drachsler,et al.  Read Between the Lines: An Annotation Tool for Multimodal Data for Learning , 2019, LAK.

[73]  Miguel Ángel Conde González,et al.  Analyzing Students' WhatsApp Messages to Evaluate the Individual Acquisition of Teamwork Competence , 2019, HCI.

[74]  Heinz Ulrich Hoppe,et al.  Enriching (Learning) Community Platforms with Learning Analytics Components , 2014, CRIWG.

[75]  Alex Rayón,et al.  Standardized enriched rubrics to support competeney-assessment through the SCALA methodology and dashboard , 2015, 2015 IEEE Global Engineering Education Conference (EDUCON).

[76]  George Siemens,et al.  Learning Analytics , 2013 .

[77]  Antoni Bibiloni,et al.  Automatic collection of user behavior in 360° multimedia , 2017, Multimedia Tools and Applications.

[78]  Christian Glahn,et al.  Using the ADL Experience API for Mobile Learning, Sensing, Informing, Encouraging, Orchestrating , 2013, 2013 Seventh International Conference on Next Generation Mobile Apps, Services and Technologies.

[79]  Ralf Klamma,et al.  Learning Analytics at Large: The Lifelong Learning Network of 160, 000 European Teachers , 2011, EC-TEL.

[80]  George Siemens,et al.  OnTask: Delivering Data-Informed, Personalized Learning Support Actions , 2018, J. Learn. Anal..

[81]  Hiroaki Ogata,et al.  A Learning Analytics Platform Approach to Seamless Learning , 2018, 2018 7th International Congress on Advanced Applied Informatics (IIAI-AAI).