Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning

Learning analytics is the analysis of electronic learning data which allows teachers, course designers and administrators of virtual learning environments to search for unobserved patterns and underlying information in learning processes. The main aim of learning analytics is to improve learning outcomes and the overall learning process in electronic learning virtual classrooms and computer-supported education. The most basic unit of learning data in virtual learning environments for learning analytics is the interaction, but there is no consensus yet on which interactions are relevant for effective learning. Drawing upon extant literature, this research defines three system-independent classifications of interactions and evaluates the relation of their components with academic performance across two different learning modalities: virtual learning environment (VLE) supported face-to-face (F2F) and online learning. In order to do so, we performed an empirical study with data from six online and two VLE-supported F2F courses. Data extraction and analysis required the development of an ad hoc tool based on the proposed interaction classification. The main finding from this research is that, for each classification, there is a relation between some type of interactions and academic performance in online courses, whereas this relation is non-significant in the case of VLE-supported F2F courses. Implications for theory and practice are discussed next.

[1]  Rod Sims,et al.  Interactivity on stage: Strategies for learner-designer communication , 1999 .

[2]  Shane Dawson,et al.  Teaching smarter: how mining ICT data can inform and improve learning and teaching practice , 2008 .

[3]  Luis Miguel,et al.  (THREE TYPES OF INTERACTION (3TI) IN ONLINE LEARNING OF CURRICULUM AND TEACHING CAPACITIES (CTCS) OF TEACHERS OF THE CANARY ISLANDS' EDUCATIONAL SYSTEM) , 2011 .

[4]  John P. Campbell,et al.  Analytics in Higher Education: Establishing a Common Language , 2012 .

[5]  Dorit Maor,et al.  Exploring Learning Analytics as Indicators of Study Behaviour , 2012 .

[6]  Jesús Favela,et al.  Informal interactions and their implications for online courses , 2004, Comput. Educ..

[7]  M. Moore.  Editorial: Three types of interaction , 1989 .

[8]  Nicola Brace,et al.  SPSS for psychologists: a guide to data analysis using SPSS for Windows (versions 9, 10 and 11). 2nd edition , 2007 .

[9]  Charalambos Vrasidas,et al.  Factors influencing interaction in an online course , 1999 .

[10]  Thomas A. Brush,et al.  Student perceptions of collaborative learning, social presence and satisfaction in a blended learning environment: Relationships and critical factors , 2008, Comput. Educ..

[11]  Atsusi Hirumi,et al.  Analysing and designing e-learning interactions , 2006 .

[12]  Charles Juwah,et al.  Interactivity in computer-mediated college and university education: A recent review of the literature , 2004, J. Educ. Technol. Soc..

[13]  Sebastián Ventura,et al.  Educational data mining: A survey from 1995 to 2005 , 2007, Expert Syst. Appl..

[14]  P. Uys.  Critical Success Factors in The Infusion of Instructional Technologies for open Learning in Development Settings: The Case of the University of Botswana , 2003 .

[15]  Roisin Donnelly,et al.  Interaction analysis in a 'Learning by Doing' problem-based professional development context , 2010, Comput. Educ..

[16]  Steven R. Malikowski,et al.  A Model for Research into Course Management Systems: Bridging Technology and Learning Theory , 2007 .

[17]  Andy P. Field,et al.  Discovering Statistics Using SPSS , 2000 .

[18]  K. Nagi,et al.  Research analysis of moodle reports to gauge the level of interactivity in elearning courses at Assumption University, Thailand , 2008, 2008 International Conference on Computer and Communication Engineering.

[19]  Julián Chaparro-Peláez,et al.  A characterisation of passive and active interactions and their influence on students' achievement using Moodle LMS logs , 2011 .

[20]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[21]  Diana Laurillard,et al.  Rethinking University Teaching: A Framework for the Effective Use of Educational Technology , 1993 .

[22]  Atsusi Hirumi,et al.  A Framework for Analyzing, Designing, and Sequencing Planned Elearning Interactions. , 2002 .

[23]  Nicola Brace,et al.  SPSS for Psychologists: A Guide to Data Analysis Using Spss for Windows (Versions 12 and 13) , 2006 .

[24]  J. Manyika.  Big data: The next frontier for innovation, competition, and productivity , 2011 .

[25]  Sunghee Shin,et al.  The effects of different instructor facilitation approaches on students' interactions during asynchronous online discussions , 2009, Comput. Educ..

[26]  Keng-Soon Soo,et al.  Interaction: What Does It Mean in Online Distance Education?. , 1998 .

[27]  Alfred P. Rovai,et al.  On-Line Course Effectiveness: An Analysis of Student Interactions and Perceptions of Learning , 2007 .

[28]  T. Anderson.  Getting the Mix Right Again: An Updated and Theoretical Rationale for Interaction , 2003 .

[29]  Daniel C. A. Hillman,et al.  Learner-Interface Interaction in Distance Education: An Extension of Contemporary Models and Strategies for Practitioners , 1994 .

[30]  Erik Duval,et al.  Dataset-driven research for improving recommender systems for learning , 2011, LAK.

[31]  Barbara Means,et al.  Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies , 2009 .

[32]  Steven Hornik,et al.  An empirical examination of factors contributing to the creation of successful e-learning environments , 2008, Int. J. Hum. Comput. Stud..

[33]  Sara McNeil,et al.  Facilitation interaction, communication and collaboration in online courses , 2000 .

[34]  L. Sutton,et al.  Vicarious Interaction: A Learning Theory for Computer-Mediated Communications , 2000 .

[35]  Mihaela Cocea,et al.  Cross-System Validation of Engagement Prediction from Log Files , 2007, EC-TEL.

[36]  Terry Anderson,et al.  Learning in a Networked World: New Roles and Responsibilties , 1998 .

[37]  Jonathan Steuer,et al.  Defining virtual reality: dimensions determining telepresence , 1992 .

[38]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[39]  Colin Beer,et al.  The indicators project identifying effective learning: Adoption, activity, grades and external factors , 2009 .

[40]  Errol Yudko,et al.  "Hits" (not "Discussion Posts") predict student success in online courses: A double cross-validation study , 2008, Comput. Educ..

[41]  Raquel Benbunan-Fich,et al.  The importance of participant interaction in online environments , 2007, Decis. Support Syst..

[42]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[43]  Griff Richards,et al.  Revisiting formative evaluation: dynamic monitoring for the improvement of learning activity design and delivery , 2011, LAK.

[44]  Shane Dawson,et al.  Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan , 2012, J. Educ. Technol. Soc..

[45]  Larry Johnson,et al.  The NMC Horizon Report: 2012 Higher Education Edition. , 2012 .