A review of ten years of implementation and research in aligning learning design with learning analytics at the Open University UK

There is an increased recognition that learning design drives both student learning experience and quality enhancements of teaching and learning. The Open University UK (OU) has been one of few institutions that have explicitly and systematically captured the designs for learning at a large scale. By applying advanced analytical techniques on large and fine-grained datasets, the OU has been unpacking the complexity of instructional practices, as well as providing conceptual and empirical evidence of how learning design influences student behaviour, satisfaction, and performance. This study discusses the implementation of learning design at the OU in the last ten years, and critically reviews empirical evidence from eight recent large-scale studies that have linked learning design with learning analytics. Four future research themes are identified to support future adoptions of learning design approaches.

[1]  Bernard Scott,et al.  Competencies for learning design: A review of the literature and a proposed framework , 2011, Br. J. Educ. Technol..

[2]  Marcelo Fabián Maina,et al.  The Art & Science of learning design , 2015 .

[3]  Donatella Persico,et al.  Informing learning design with learning analytics to improve teacher inquiry , 2015, Br. J. Educ. Technol..

[4]  Rob Koper,et al.  Educational modelling language: modelling reusable, interoperable, rich and personalised units of learning , 2004, Br. J. Educ. Technol..

[5]  Bart Rienties,et al.  Understanding academics’ resistance towards (online) student evaluation , 2014 .

[6]  Michael Derntl,et al.  Handbook of research on learning design and learning objects: issues, applications and technologies , 2009 .

[7]  Julie L. Booth,et al.  Instructional Complexity and the Science to Constrain It , 2013, Science.

[8]  Bart Rienties,et al.  Learning Design – creative design to visualise learning activities , 2016 .

[9]  A. Wise,et al.  Why Theory Matters More than Ever in the Age of Big Data , 2015, J. Learn. Anal..

[10]  Barbara Wasson,et al.  Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action , 2015, Br. J. Educ. Technol..

[11]  Shane Dawson,et al.  Informing Pedagogical Action , 2013 .

[12]  Bart Rienties,et al.  "Scaling up" learning design: impact of learning design activities on LMS behavior and performance , 2015, LAK.

[13]  Bart Rienties,et al.  Modelling and Managing Learner Satisfaction: Use of Learner Feedback to Enhance Blended and Online Learning Experience , 2016 .

[14]  D. Carless Differing perceptions in the feedback process , 2006 .

[15]  Uta Boehm,et al.  Designing For Learning In An Open World , 2016 .

[16]  Bart Rienties,et al.  Mixing and Matching Learning Design and Learning Analytics , 2017, HCI.

[17]  Alejandro Armellini,et al.  Learning design and assessment with e-tivities , 2010, Br. J. Educ. Technol..

[18]  Bart Rienties,et al.  The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules , 2016, Comput. Hum. Behav..

[19]  J. B. Arbaugh,et al.  System, scholar or students? Which most influences online MBA course effectiveness? , 2014, J. Comput. Assist. Learn..

[20]  Zdenek Zdráhal,et al.  Implementing predictive learning analytics on a large scale: the teacher's perspective , 2017, LAK.

[21]  Martin Weller,et al.  Final Project Report of the OULDI- JISC Project: Challenge and Change in Curriculum Design Process, Communities, Visualisation and Practice , 2012 .

[22]  Dirk T. Tempelaar,et al.  In search for the most informative data for feedback generation: Learning analytics in a data-rich context , 2015, Comput. Hum. Behav..

[23]  Marek Hatala,et al.  Learning at distance: Effects of interaction traces on academic achievement , 2015, Comput. Educ..

[24]  Eva Dobozy,et al.  The Larnaca Declaration on Learning Design , 2016 .

[25]  Bart Rienties,et al.  The impact of 151 learning designs on student satisfaction and performance: social learning (analytics) matters , 2016, LAK.

[26]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..

[27]  Kathleen M. T. Collins,et al.  Students’ Perceptions of Characteristics of Effective College Teachers: A Validity Study of a Teaching Evaluation Form Using a Mixed-Methods Analysis , 2007 .

[28]  Bart Rienties,et al.  Unravelling the dynamics of instructional practice: a longitudinal study on learning design and VLE activities , 2017, LAK.

[29]  Shane Dawson,et al.  Learning designs and learning analytics , 2011, LAK.

[30]  Bart Rienties,et al.  Making sense of learner and learning Big Data: reviewing five years of Data Wrangling at the Open University UK , 2017 .

[31]  James Dalziel,et al.  Learning Design: Conceptualizing a Framework for Teaching and Learning Online , 2014 .

[32]  Denise Whitelock,et al.  #Design4Learning: Designing for the Future of Higher Education , 2016 .

[33]  Bart Rienties,et al.  Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making , 2016, Br. J. Educ. Technol..

[34]  Diana Laurillard,et al.  Virtual Teaching Tools : Bringing academics closer to the design of e-learning , 2002 .

[35]  Sue Bennett,et al.  Technology tools to support learning design: Implications derived from an investigation of university teachers' design practices , 2015, Comput. Educ..

[36]  George Siemens,et al.  Let’s not forget: Learning analytics are about learning , 2015 .

[37]  Bart Rienties,et al.  Scholarly insight 2016: a Data wrangler perspective , 2016 .

[38]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[39]  Bart Rienties,et al.  Assessing Learning Gains , 2016, TEA.

[40]  Linda Corrin,et al.  A conceptual framework linking learning design with learning analytics , 2016, LAK.

[41]  Daniela Wong Gonzalez,et al.  Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related , 2017 .

[42]  James Dalziel,et al.  Implementing learning design : the Learning Activity Management System (LAMS) , 2003 .

[43]  Rebecca Ferguson,et al.  Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates , 2017, Comput. Hum. Behav..

[44]  J. Beishuizen,et al.  Student learning experience as indicator of teaching quality , 2012 .

[45]  G. Conole,et al.  Visualising learning design to foster and support good practice and creativity , 2008 .

[46]  Richard Hays,et al.  Evaluating teaching and learning , 2018 .

[47]  B. Rienties,et al.  Online learning experiences of new versus continuing learners: a large-scale replication study , 2017 .

[48]  Rebecca Ferguson,et al.  Innovating Pedagogy 2016: Open University Innovation Report 5 , 2016 .

[49]  Doug Clow An overview of learning analytics , 2013 .

[50]  Bart Rienties,et al.  Analytics in online and offline language learning environments: the role of learning design to understand student online engagement , 2018 .

[51]  Gráinne Conole,et al.  Designing for Learning in an Open World , 2012 .

[52]  Rebecca Ferguson,et al.  Social learning analytics: five approaches , 2012, LAK.