The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules

Pedagogically informed designs of learning are increasingly of interest to researchers in blended and online learning, as learning design is shown to have an impact on student behaviour and outcomes. Although learning design is widely studied, often these studies are individual courses or programmes and few empirical studies have connected learning designs of a substantial number of courses with learning behaviour. In this study we linked 151 modules and 111.256 students with students' behaviour (<400 million minutes of online behaviour), satisfaction and performance at the Open University UK using multiple regression models. Our findings strongly indicate the importance of learning design in predicting and understanding Virtual Learning Environment behaviour and performance of students in blended and online environments. In line with proponents of social learning theories, our primary predictor for academic retention was the time learners spent on communication activities, controlling for various institutional and disciplinary factors. Where possible, appropriate and well designed communication tasks that align with the learning objectives of the course may be a way forward to enhance academic retention. Pedagogically informed learning designs (LD) are increasingly of interest.Few empirical studies have connected LD with behaviour, satisfaction and retention.Using regression analyses we linked LDs of 151 modules and 111?K students.LD has strong impact on behaviour, satisfaction, and performance.Primary predictor for academic retention was communication activities.

[1]  Bernard Scott,et al.  Competencies for learning design: A review of the literature and a proposed framework , 2011, Br. J. Educ. Technol..

[2]  J. Beishuizen,et al.  Student learning experience as indicator of teaching quality , 2012 .

[3]  J. Arbaugh,et al.  Technological and Structural Characteristics, Student Learning and Satisfaction with Web-Based Courses , 2002 .

[4]  Albert W. M. Hoogveld,et al.  Application of an instructional systems design approach by teachers in higher education: individual versus team design , 2003 .

[5]  Dirk T. Tempelaar,et al.  Investigating the relations between motivation, tool use, participation, and performance in an e-learning course using web-videoconferencing , 2013, Comput. Hum. Behav..

[6]  Paul D. Rudman,et al.  Designing for learning in an open world , 2013 .

[7]  Bart Rienties,et al.  Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making , 2016, Br. J. Educ. Technol..

[8]  Julie L. Booth,et al.  Instructional Complexity and the Science to Constrain It , 2013, Science.

[9]  Rebecca Ferguson,et al.  Social Learning Analytics , 2012, J. Educ. Technol. Soc..

[10]  Bart Rienties,et al.  "Scaling up" learning design: impact of learning design activities on LMS behavior and performance , 2015, LAK.

[11]  Bart Rienties,et al.  Modelling and Managing Learner Satisfaction: Use of Learner Feedback to Enhance Blended and Online Learning Experience , 2016 .

[12]  J. B. Arbaugh,et al.  Is There an Optimal Design for On-Line MBA Courses? , 2005 .

[13]  Alejandro Armellini,et al.  Learning design and assessment with e-tivities , 2010, Br. J. Educ. Technol..

[14]  Davinia Hernández Leo,et al.  LdShake support for team-based learning design , 2014, Comput. Hum. Behav..

[15]  Martin Weller,et al.  Final Project Report of the OULDI- JISC Project: Challenge and Change in Curriculum Design Process, Communities, Visualisation and Practice , 2012 .

[16]  Dirk T. Tempelaar,et al.  In search for the most informative data for feedback generation: Learning analytics in a data-rich context , 2015, Comput. Hum. Behav..

[17]  Miguel Ángel Conde González,et al.  Learning analytics for educational decision making , 2015, Comput. Hum. Behav..

[18]  J. B. Arbaugh,et al.  System, scholar or students? Which most influences online MBA course effectiveness? , 2014, J. Comput. Assist. Learn..

[19]  Carolyn Penstein Rosé,et al.  Learning analytics and machine learning , 2014, LAK.

[20]  Iván Martínez-Ortiz,et al.  Educational game design for online education , 2008, Comput. Hum. Behav..

[21]  Rebecca Ferguson,et al.  Social learning analytics: five approaches , 2012, LAK.

[22]  Pieter Wouters,et al.  Learner Performance in Multimedia Learning Arrangements: An Analysis Across Instructional Approaches , 2009 .

[23]  Carol Calvert,et al.  Developing a model and applications for probabilities of student success: a case study of predictive analytics , 2014 .

[24]  Richard E. Clark,et al.  Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching , 2006 .

[25]  M. Moore Editorial: Three types of interaction , 1989 .

[26]  Lev Vygotsky Mind in society , 1978 .

[27]  Kathleen M. T. Collins,et al.  Students’ Perceptions of Characteristics of Effective College Teachers: A Validity Study of a Teaching Evaluation Form Using a Mixed-Methods Analysis , 2007 .

[28]  Dirk T. Tempelaar,et al.  A review of the role of information communication technology and course design in transitional education practices , 2012, Interact. Learn. Environ..

[29]  Daniel T. Hickey,et al.  Small to big before massive: scaling up participatory learning analytics , 2014, LAK.

[30]  Carolyn Penstein Rosé,et al.  Social factors that contribute to attrition in MOOCs , 2014, L@S.

[31]  J. Arbaugh,et al.  A Structural Equation Model of Predictors for Effective Online Learning , 2005 .

[32]  John T. E. Richardson,et al.  Approaches to studying across the adult life span: Evidence from distance education , 2013 .