A Mixed-Method Study of How Instructors Design for Learning in Online and Distance Education

The use of analytical methods from learning analytics research combined with visualisations of learning activities using learning design tools and frameworks has provided important insights into how instructors design for learning. Nonetheless, there are many subtle nuances in instructors’ design decisions that might not be easily be captured using learning analytics tools. Therefore, this study sets out to explore how and why instructors design for learning in an online and distance higher education setting by employing a mixed-method approach, which combined semi-structured interviews of 12 instructors with network analyses of their learning designs. Our findings uncovered several underlying factors that influenced how instructors designed their modules, and highlighted some discrepancies between instructors’ pedagogical beliefs and their actual learning design as captured by OULDI. This study showcased the potential of combining learning analytics with qualitative insights for a better understanding of the complex design process in online distance higher education.

[1]  Denise Whitelock,et al.  Student workload: a case study of its significance, evaluation and management at the Open University , 2015 .

[2]  Mary Thorpe,et al.  Rethinking Learner Support: The challenge of collaborative online learning , 2002 .

[3]  Martin Weller,et al.  Final Project Report of the OULDI- JISC Project: Challenge and Change in Curriculum Design Process, Communities, Visualisation and Practice , 2012 .

[4]  Linda Corrin,et al.  A conceptual framework linking learning design with learning analytics , 2016, LAK.

[5]  Davinia Hernández Leo,et al.  An Integrated Environment for Learning Design , 2018, Front. ICT.

[6]  Rebecca Ferguson,et al.  Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates , 2017, Comput. Hum. Behav..

[7]  Bart Rienties,et al.  Unravelling the dynamics of instructional practice: a longitudinal study on learning design and VLE activities , 2017, LAK.

[8]  Thomas Daniel Ullmann,et al.  Automated Analysis of Reflection in Writing: Validating Machine Learning Approaches , 2019, International Journal of Artificial Intelligence in Education.

[9]  Alyssa Friend Wise,et al.  Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling , 2017, Internet High. Educ..

[10]  J. Creswell.  Qualitative inquiry and research design: Choosing among five approaches, 2nd ed. , 2007 .

[11]  Michail N. Giannakos,et al.  Learning Analytics for Learning Design: A Systematic Literature Review of Analytics-Driven Design to Enhance Learning , 2019, IEEE Transactions on Learning Technologies.

[12]  Barbara Wasson,et al.  Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action , 2015, Br. J. Educ. Technol..

[13]  Rob Koper,et al.  Educational modelling language: modelling reusable, interoperable, rich and personalised units of learning , 2004, Br. J. Educ. Technol..

[14]  James Dalziel,et al.  Implementing learning design : the Learning Activity Management System (LAMS) , 2003 .

[15]  Maura R. Cherney,et al.  Online Course Student Collaboration Literature: A Review and Critique , 2018 .

[16]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[17]  Shane Dawson,et al.  Learning designs and learning analytics , 2011, LAK.

[18]  Eva Dobozy,et al.  Informing Learning Design through Analytics: Applying Network Graph Analysis. , 2018 .

[19]  Dirk T. Tempelaar,et al.  Overcoming cross-cultural group work tensions: mixed student perspectives on the role of social relationships , 2018 .

[20]  Gráinne Conole,et al.  Designing for Learning in an Open World , 2012 .

[21]  Paul A. Kirschner,et al.  Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: a review of the research , 2003, Comput. Hum. Behav..

[22]  Bart Rienties,et al.  "Scaling up" learning design: impact of learning design activities on LMS behavior and performance , 2015, LAK.

[23]  Ellie Chambers.  Work-load and the quality of student learning , 1992 .

[24]  Bart Rienties,et al.  The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules , 2016, Comput. Hum. Behav..

[25]  Bart Rienties,et al.  Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making , 2016, Br. J. Educ. Technol..

[26]  Ling Li,et al.  A Pattern Language Based Learning Design Studio for an Analytics Informed Inter-Professional Design Community , 2017, IxD&A.

[27]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[28]  Patricia Charlton,et al.  Using technology to develop teachers as designers of TEL: Evaluating the learning designer , 2018, Br. J. Educ. Technol..

[29]  B. Rienties,et al.  How do students engage with computer-based assessments: impact of study breaks on intertemporal engagement and pass rates , 2018 .

[30]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..

[31]  M. Newman,et al.  Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[32]  Donatella Persico,et al.  Teachers as designers of TEL interventions , 2018, Br. J. Educ. Technol..

[33]  James Dalziel,et al.  Learning Design: Conceptualizing a Framework for Teaching and Learning Online , 2014 .

[34]  Dai Griffiths,et al.  The Use of Models in Learning Design and Learning Analytics , 2017, IxD&A.

[35]  Donatella Persico,et al.  Informing learning design with learning analytics to improve teacher inquiry , 2015, Br. J. Educ. Technol..