A Multilevel Framework and Method for Learning Analytics Integrated Learning Design

Efforts to realize the potential of learning analytics (LA) to contribute to improving student learning and learning design have brought important advances. A review of successful cases of learning analytics applications reveals that 1) there is a tight coupling between the learning outcome (LO) goals, task sequence design, and the learning analytics and feedback in each case, and 2) the learning analytics to be deployed and the feedback to be provided to learners and/or teachers are integral to the learning design (LD) rather than constructed after the LD is completed. Learning design frameworks in the literature have focused on generic learning task taxonomies and are unable to scaffold LA-integrated LD practice. This paper proposes a multilevel framework for LA-integrated LD, which provides a hierarchically nested multilevel structure for the design of LD and LA elements based on 60 STEM curriculum units collected from authentic classrooms. The framework includes a design process model in the form of a Learning Design Triangle and the concept of Learning Analytics integrated Curriculum Component Design Patterns (LA-CCDP). Operationalization of the framework is illustrated using one STEM curriculum unit. This framework can be adopted for professional learning and technology development to support LA-integrated LD practices.

[1]  Ling Li,et al.  A Pattern Language Based Learning Design Studio for an Analytics Informed Inter-Professional Design Community , 2017, IxD&A.

[2]  L. Earl,et al.  Assessment as Learning: Using Classroom Assessment to Maximize Student Learning , 2003 .

[3]  Dennis P. Groth,et al.  Engaging Faculty in Learning Analytics: Agents of Institutional Culture Change , 2019, J. Learn. Anal..

[4]  Paul Salvador Inventado,et al.  Data-driven design pattern production: a case study on the ASSISTments online learning system , 2015, EuroPLoP.

[5]  Sabine Seufert,et al.  A Pedagogical Perspective on Big Data and Learning Analytics: A Conceptual Model for Digital Learning Support , 2019, Technology, Knowledge and Learning.

[6]  C. Hmelo‐Silver.  Problem-Based Learning: What and How Do Students Learn? , 2004 .

[7]  Manolis Mavrikis,et al.  Intelligent analysis and data visualisation for teacher assistance tools: The case of exploratory learning , 2019, Br. J. Educ. Technol..

[8]  June Ahn,et al.  Designing in Context: Reaching Beyond Usability in Learning Analytics Dashboard Design , 2019, J. Learn. Anal..

[9]  Carolyn Penstein Rosé,et al.  Explanatory learner models: Why machine learning (alone) is not the answer , 2019, Br. J. Educ. Technol..

[10]  Alejandro Peña Ayala,et al.  Learning analytics: A glance of evolution, status, and trends according to a proposed taxonomy , 2018, WIREs Data Mining Knowl. Discov..

[11]  James Dalziel,et al.  Implementing learning design : a decade of lessons learned , 2013 .

[12]  Patricia Charlton,et al.  Enabling creative learning design through semantic technologies , 2012 .

[13]  Rebecca Ferguson,et al.  Human-Centred Learning Analytics , 2019, J. Learn. Anal..

[14]  Brock Craft,et al.  Using the Learning Designer to develop a conceptual framework for linking learning design tools and systems , 2011 .

[15]  Heinz Ulrich Hoppe,et al.  Analysing content and patterns of interaction for improving the learning design of networked learning environments , 2015, Br. J. Educ. Technol..

[16]  Thomas R. Gruber,et al.  Toward principles for the design of ontologies used for knowledge sharing? , 1995, Int. J. Hum. Comput. Stud..

[17]  Shane Dawson,et al.  Where learning analytics meets learning design , 2012, LAK '12.

[18]  Ran Liu,et al.  Closing the loop: Automated data-driven cognitive model discoveries lead to improved instruction and learning , 2017, EDM.

[19]  Peter Goodyear,et al.  Educational design and networked learning: Patterns, pattern languages and design practice , 2005 .

[20]  María Jesús Rodríguez-Triana,et al.  Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom level , 2019, Australasian Journal of Educational Technology.

[21]  Matthew J. Koehler,et al.  Technological Pedagogical Content Knowledge (TPACK) , 2009 .

[22]  Bart Rienties,et al.  The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules , 2016, Comput. Hum. Behav..

[23]  Margery H. Davis,et al.  Outcome-based education. , 2003, Journal of veterinary medical education.

[24]  Tom Boyle.  Layered learning design: Towards an integration of learning design and learning object perspectives , 2010, Comput. Educ..

[25]  H S Barrows,et al.  A taxonomy of problem‐based learning methods , 1986, Medical education.

[26]  D. Randy Garrison,et al.  Self-Directed and Self-Regulated Learning: Conceptual Links , 2013 .

[27]  Jill Willis,et al.  A cognitive processing framework for learning analytics , 2014, LAK.

[28]  Kenneth R. Koedinger,et al.  Using Data-Driven Discovery of Better Student Models to Improve Student Learning , 2013, AIED.

[29]  Simon Knight,et al.  Design and implementation of a pedagogic intervention using writing analytics , 2017 .

[30]  Bart Rienties,et al.  A review of ten years of implementation and research in aligning learning design with learning analytics at the Open University UK , 2017, IxD&A.

[31]  Simon Buckingham Shum,et al.  Contextualizable Learning Analytics Design: A Generic Model and Writing Analytics Evaluations , 2019, LAK.

[32]  Max Jacobson,et al.  A Pattern Language: Towns, Buildings, Construction , 1981 .

[33]  Shane Dawson,et al.  Informing Pedagogical Action , 2013 .

[34]  Bart Rienties,et al.  Learning Design – creative design to visualise learning activities , 2016 .

[35]  Michail N. Giannakos,et al.  Learning Analytics for Learning Design: A Systematic Literature Review of Analytics-Driven Design to Enhance Learning , 2019, IEEE Transactions on Learning Technologies.

[36]  Bart Rienties,et al.  "Scaling up" learning design: impact of learning design activities on LMS behavior and performance , 2015, LAK.

[37]  C. Shuler,et al.  Comparisons in Basic Science Learning Outcomes Between Students in PBL and Traditional Dental Curricula at the Same Dental School , 2012 .