Unravelling the dynamics of instructional practice: a longitudinal study on learning design and VLE activities

Substantial progress has been made in understanding how teachers design for learning. However, there remains a paucity of evidence of the actual students' response towards leaning designs. Learning analytics has the power to provide just-in-time support, especially when predictive analytics is married with the way teachers have designed their course, or so-called a learning design. This study investigates how learning designs are configured over time and their impact on student activities by analyzing longitudinal data of 38 modules with a total of 43,099 registered students over 30 weeks at the Open University UK, using social network analysis and panel data analysis. Our analysis unpacked dynamic configurations of learning designs between modules over time, which allows teachers to reflect on their practice in order to anticipate problems and make informed interventions. Furthermore, by controlling for the heterogeneity between modules, our results indicated that learning designs were able to explain up to 60% of the variability in student online activities, which reinforced the importance of pedagogical context in learning analytics.

[1]  Shane Dawson,et al.  SNAPP: a bird's-eye view of temporal participant interaction , 2011, LAK.

[2]  Rob Koper,et al.  Educational modelling language: modelling reusable, interoperable, rich and personalised units of learning , 2004, Br. J. Educ. Technol..

[3]  Martin Weller,et al.  Final Project Report of the OULDI- JISC Project: Challenge and Change in Curriculum Design Process, Communities, Visualisation and Practice , 2012 .

[4]  References , 1971 .

[5]  Bernard Scott,et al.  Competencies for learning design: A review of the literature and a proposed framework , 2011, Br. J. Educ. Technol..

[6]  Bart Rienties,et al.  Learning Design – creative design to visualise learning activities , 2016 .

[7]  A. Wise,et al.  Why Theory Matters More than Ever in the Age of Big Data , 2015, J. Learn. Anal..

[8]  Dirk T. Tempelaar,et al.  In search for the most informative data for feedback generation: Learning analytics in a data-rich context , 2015, Comput. Hum. Behav..

[9]  Barbara Wasson,et al.  Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action , 2015, Br. J. Educ. Technol..

[10]  Bart Rienties,et al.  "Scaling up" learning design: impact of learning design activities on LMS behavior and performance , 2015, LAK.

[11]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[12]  Linda Corrin,et al.  A conceptual framework linking learning design with learning analytics , 2016, LAK.

[13]  Richard Halverson,et al.  Systems of Practice: How Leaders Use Artifacts to Create Professional Community in Schools , 2003 .

[14]  Xin Chen,et al.  "Twitter Archeology" of learning analytics and knowledge conferences , 2015, LAK.

[15]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[16]  Marek Hatala,et al.  Learning at distance: Effects of interaction traces on academic achievement , 2015, Comput. Educ..

[17]  Donatella Persico,et al.  Informing learning design with learning analytics to improve teacher inquiry , 2015, Br. J. Educ. Technol..

[18]  Ray Faulkner,et al.  The Teaching of Design , 1948 .

[19]  Matthew T. Hora,et al.  Instructional Systems of Practice: A Multidimensional Analysis of Math and Science Undergraduate Course Planning and Classroom Teaching , 2013 .

[20]  Bart Rienties,et al.  The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules , 2016, Comput. Hum. Behav..

[21]  Marek Hatala,et al.  What do cMOOC participants talk about in social media?: a topic analysis of discourse in a cMOOC , 2015, LAK.

[22]  Dirk T. Tempelaar,et al.  What learning analytics based prediction models tell us about feedback preferences of students , 2016 .

[23]  Doug Clow An overview of learning analytics , 2013 .

[24]  M. Newman,et al.  Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Pierre Rochus,et al.  Hu, Luojia , Estimation of a censored dynamic panel data model,Econometrica. Journal of the Econometric Society , 2002 .

[26]  Shane Dawson,et al.  Informing Pedagogical Action , 2013 .

[27]  Darryl J. Downing,et al.  Univariate Tests for Time Series Models , 1993 .

[28]  D. Boud,et al.  Aligning assessment with long‐term learning , 2006 .

[29]  Barry Harper,et al.  Handbook of Research on Learning Design and Learning Objects: Issues, Applications and Technologies , 2008 .

[30]  Á. M. Hernáez Borgatti, Stephen; Martin Everett i Lin Freeman. UCINET IV. Network Analysis Software. Version 1.0. Columbia: Analytic Technologies, 1992 , 1995 .

[31]  Simon Buckingham Shum,et al.  Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics , 2012, International Conference on Learning Analytics and Knowledge.

[32]  Karina L. Cela,et al.  Social Network Analysis in E-Learning Environments: A Preliminary Systematic Review , 2014, Educational Psychology Review.

[33]  Shane Dawson,et al.  Untangling MOOC learner networks , 2016, LAK.

[34]  Eva Dobozy,et al.  Transdisciplinary Pedagogical Templates and Their Potential for Adaptive Reuse. , 2016 .

[35]  Arthur C. Graesser,et al.  How do you connect?: analysis of social capital accumulation in connectivist MOOCs , 2015, LAK.

[36]  Gráinne Conole,et al.  Designing for Learning in an Open World , 2012 .

[37]  Stephen G. Pulman,et al.  OpenEssayist: a supply and demand learning analytics tool for drafting academic essays , 2015, LAK.

[38]  Shane Dawson,et al.  Learning designs and learning analytics , 2011, LAK.

[39]  J. Hausman Specification tests in econometrics , 1978 .

[40]  Bart Rienties,et al.  Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making , 2016, Br. J. Educ. Technol..

[41]  Mengxiao Zhu,et al.  An exploratory study using social network analysis to model eye movements in mathematics problem solving , 2015, LAK.

[42]  James Dalziel,et al.  Learning Design: Conceptualizing a Framework for Teaching and Learning Online , 2014 .

[43]  Sue Bennett,et al.  Technology tools to support learning design: Implications derived from an investigation of university teachers' design practices , 2015, Comput. Educ..

[44]  George Siemens,et al.  Let’s not forget: Learning analytics are about learning , 2015 .

[45]  Tobias Hecking,et al.  Investigating social and semantic user roles in MOOC discussion forums , 2016, LAK.

[46]  James Dalziel,et al.  Implementing learning design : the Learning Activity Management System (LAMS) , 2003 .

[47]  David Coniam,et al.  Continuing Education Reform in Hong Kong: Issues of Contextualization. , 2003 .

[48]  Alyssa Friend Wise,et al.  Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling , 2017, Internet High. Educ..

[49]  Diana Laurillard,et al.  Virtual Teaching Tools : Bringing academics closer to the design of e-learning , 2002 .

[50]  Bart Rienties,et al.  Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK , 2016 .

[51]  Eva Dobozy,et al.  The Larnaca Declaration on Learning Design , 2016 .

[52]  Martin Hlosta,et al.  OU Analyse: analysing at-risk students at The Open University , 2015 .

[53]  Bart Rienties,et al.  The impact of 151 learning designs on student satisfaction and performance: social learning (analytics) matters , 2016, LAK.

[54]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..