Developing early warning systems to predict students' online learning performance

We develop early warning systems to predict at-risk students while a course is in progress.Learning portfolios from a fully online course are evaluated by data mining techniques.The results show that CART supplemented by AdaBoost has the best classification performance.Time-dependent variables are essential to identify student online learning performance. An early warning system can help to identify at-risk students, or predict student learning performance by analyzing learning portfolios recorded in a learning management system (LMS). Although previous studies have shown the applicability of determining learner behaviors from an LMS, most investigated datasets are not assembled from online learning courses or from whole learning activities undertaken on courses that can be analyzed to evaluate students' academic achievement. Previous studies generally focus on the construction of predictors for learner performance evaluation after a course has ended, and neglect the practical value of an "early warning" system to predict at-risk students while a course is in progress. We collected the complete learning activities of an online undergraduate course and applied data-mining techniques to develop an early warning system. Our results showed that, time-dependent variables extracted from LMS are critical factors for online learning. After students have used an LMS for a period of time, our early warning system effectively characterizes their current learning performance. Data-mining techniques are useful in the construction of early warning systems; based on our experimental results, classification and regression tree (CART), supplemented by AdaBoost is the best classifier for the evaluation of learning performance investigated by this study.

[1]  Alvin Y. Wang,et al.  Predictors of Performance in the Virtual Classroom: Identifying and Helping At-Risk Cyber-Students , 2002 .

[2]  Martin Muehlenbrock.  Automatic Action Analysis in an Interactive Learning Environment , 2005 .

[3]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[4]  Gwo-Jen Hwang,et al.  A Computer-Assisted Approach to Diagnosing Student Learning Problems in Science courses , 2003, J. Inf. Sci. Eng..

[5]  Sotiris B. Kotsiantis.  Use of machine learning techniques for educational proposes: a decision support system for forecasting students’ grades , 2011, Artificial Intelligence Review.

[6]  Hüseyin Gürüler,et al.  A new student performance analysing system using knowledge discovery in higher educational databases , 2010, Comput. Educ..

[7]  Sotiris B. Kotsiantis,et al.  Predicting students marks in Hellenic Open University , 2005, Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05).

[8]  E. Sadler‐Smith.  The relationship between learning style and cognitive style , 2001 .

[9]  Eibe Frank,et al.  Speeding Up Logistic Model Tree Induction , 2005, PKDD.

[10]  Chih-Ming Chen,et al.  Mining Key Formative Assessment Rules based on Learner Profiles for Web-based Learning Systems , 2007, Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007).

[11]  Geraldine Clarebout,et al.  Students' tool-use within a web enhanced course: Explanatory mechanisms of students' tool-use pattern , 2013, Comput. Hum. Behav..

[12]  Àngela Nebot,et al.  Applying Data Mining Techniques to e-Learning Problems , 2007 .

[13]  J. Beck,et al.  An Educational Data Mining Tool to Browse Tutor-Student Interactions : Time Will Tell ! , 2005 .

[14]  M. Hanna.  Data mining in the e‐learning domain , 2004 .

[15]  Haruna Chiroma,et al.  Data Mining for Education Decision Support: A Review , 2014, Int. J. Emerg. Technol. Learn..

[16]  Elena Gaudioso,et al.  Data mining to support tutoring in virtual learning communities: experiences and challenges , 2005 .

[17]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[18]  Philip J. Goldstein,et al.  Academic Analytics : The Uses of Management Information and Technology in Higher Education , 2005 .

[19]  Ramakrishnan Srikant,et al.  Fast algorithms for mining association rules , 1998, VLDB 1998.

[20]  Sotiris B. Kotsiantis,et al.  PREDICTING STUDENTS' PERFORMANCE IN DISTANCE LEARNING USING MACHINE LEARNING TECHNIQUES , 2004, Appl. Artif. Intell..

[21]  Jack Mostow,et al.  A Generic Tool to Browse Tutor-Student Interactions: Time Will Tell! , 2005, AIED.

[22]  Paulo J. G. Lisboa,et al.  Learning what is important: feature selection and rule extraction in a virtual course , 2006, ESANN.

[23]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[24]  Fu-Yun Yu,et al.  Predictive Effects of Online Peer Feedback Types on Performance Quality , 2013, J. Educ. Technol. Soc..

[25]  W. F. Punch,et al.  Predicting student performance: an application of data mining methods with an educational Web-based system , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[26]  Margaret H. Dunham,et al.  Data Mining: Introductory and Advanced Topics , 2002 .

[27]  I. D. Zaharakis,et al.  EFFICIENCY OF MACHINE LEARNING TECHNIQUES IN PREDICTING STUDENTS ’ PERFORMANCE IN DISTANCE LEARNING SYSTEMS , 2005 .

[28]  Manuel de Buenaga Rodríguez,et al.  An intelligent information access system assisting a case based learning methodology evaluated in higher education with medical students , 2012, Comput. Educ..

[29]  Eva Lucrecia Gibaja Galindo,et al.  Predicting students' marks from Moodle logs using neural network models , 2006 .

[30]  Hanan Ayad,et al.  Student success system: risk analytics and data visualization using ensembles of predictive models , 2012, LAK.

[31]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[32]  Chong Ho Yu,et al.  Using SAS/IntrNet for Evaluating Web‐based Courses , 1998 .

[33]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[34]  Ramakrishnan Srikant,et al.  Fast Algorithms for Mining Association Rules in Large Databases , 1994, VLDB.

[35]  William F. Punch,et al.  Mining interesting contrast rules for a web-based educational system , 2004, 2004 International Conference on Machine Learning and Applications, 2004. Proceedings..

[36]  Lida Xu,et al.  Business Intelligence for Enterprise Systems: A Survey , 2012, IEEE Transactions on Industrial Informatics.

[37]  Laurie P. Dringus,et al.  Using data mining as a strategy for assessing asynchronous discussion forums , 2005, Comput. Educ..

[38]  Stavros Valsamidis,et al.  A Clustering Methodology of Web Log Data for Learning Management Systems , 2012, J. Educ. Technol. Soc..

[39]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[40]  Sotiris B. Kotsiantis,et al.  Preventing Student Dropout in Distance Learning Using Machine Learning Techniques , 2003, KES.

[41]  Brian W. Junker,et al.  Predicting end-of-year accountability assessment scores from monthly student records in an online tutoring system , 2006 .

[42]  Sebastián Ventura,et al.  Data mining in course management systems: Moodle case study and tutorial , 2008, Comput. Educ..

[43]  Alvin Y. Wang,et al.  A Discourse Analysis of Online Classroom Chats: Predictors of Cyber-Student Performance , 2001 .

[44]  Félix Hernández-del-Olmo,et al.  Supporting teachers in adaptive educational systems through predictive models: A proof of concept , 2012, Expert Syst. Appl..

[45]  Wu He,et al.  Examining students' online interaction in a live video streaming environment using data mining and text mining , 2013, Comput. Hum. Behav..

[46]  Kathryn Nantz,et al.  The learning portfolio: Reflective practice for improving student learning 2nd Ed. , 2009 .

[47]  Chih-Ping Chu,et al.  A learning style classification mechanism for e-learning , 2009, Comput. Educ..