Participation-based student final performance prediction model through interpretable Genetic Programming: Integrating learning analytics, educational data mining and theory

A theory-based method for a computational student performance prediction.A Genetic Programming model for grade prediction is described and tested.Model evaluation suggests high success rates for predicting student grades. Building a student performance prediction model that is both practical and understandable for users is a challenging task fraught with confounding factors to collect and measure. Most current prediction models are difficult for teachers to interpret. This poses significant problems for model use (e.g. personalizing education and intervention) as well as model evaluation. In this paper, we synthesize learning analytics approaches, educational data mining (EDM) and HCI theory to explore the development of more usable prediction models and prediction model representations using data from a collaborative geometry problem solving environment: Virtual Math Teams with Geogebra (VMTwG). First, based on theory proposed by Hrastinski (2009) establishing online learning as online participation, we operationalized activity theory to holistically quantify students' participation in the CSCL (Computer-supported Collaborative Learning) course. As a result, 6 variables, Subject, Rules, Tools, Division of Labor, Community, and Object, are constructed. This analysis of variables prior to the application of a model distinguishes our approach from prior approaches (feature selection, Ad-hoc guesswork etc.). The approach described diminishes data dimensionality and systematically contextualizes data in a semantic background. Secondly, an advanced modeling technique, Genetic Programming (GP), underlies the developed prediction model. We demonstrate how connecting the structure of VMTwG trace data to a theoretical framework and processing that data using the GP algorithmic approach outperforms traditional models in prediction rate and interpretability. Theoretical and practical implications are then discussed.

[1]  John P. Campbell,et al.  Academic Analytics: A New Tool for a New Era. , 2007 .

[2]  Wei Wang,et al.  A Genetic Programming Model for Real-Time Crash Prediction on Freeways , 2013, IEEE Transactions on Intelligent Transportation Systems.

[3]  Robert D. Rodman,et al.  An Introduction to Language , 1984 .

[4]  John R. Koza,et al.  Genetic Programming II , 1992 .

[5]  Sean P. Goggins,et al.  Social Ability in Online Groups: Representing the Quality of Interactions in Social Computing Environments , 2009, 2009 International Conference on Computational Science and Engineering.

[6]  Ron Kohavi,et al.  Guest Editors' Introduction: On Applied Research in Machine Learning , 1998, Machine Learning.

[7]  Gail L. Sunderman,et al.  Predictable Failure of Federal Sanctions-Driven Accountability for School Improvement—And Why We May Retain It Anyway , 2009 .

[8]  Allyson Hadwin,et al.  Measurement and assessment in computer-supported collaborative learning , 2010, Comput. Hum. Behav..

[9]  Sean P. Goggins,et al.  Learning analytics in CSCL with a focus on assessment: an exploratory study of activity theory-informed cluster analysis , 2014, LAK.

[10]  César Hervás-Martínez,et al.  Data Mining Algorithms to Classify Students , 2008, EDM.

[11]  Hani Hagras,et al.  A genetic type-2 fuzzy logic based system for the generation of summarised linguistic predictive models for financial applications , 2013, Soft Comput..

[12]  Sean P. Goggins,et al.  Group informatics: A methodological approach and ontology for sociotechnical group research , 2013, J. Assoc. Inf. Sci. Technol..

[13]  George Siemens,et al.  Learning analytics and educational data mining: towards communication and collaboration , 2012, LAK.

[14]  Frank E. Harrell,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2001 .

[15]  Siddhartha Bhattacharyya,et al.  Genetic programming in classifying large-scale data: an ensemble method , 2004, Inf. Sci..

[16]  Olga Basharina AN ACTIVITY THEORY PERSPECTIVE ON STUDENT-REPORTED CONTRADICTIONS IN INTERNATIONAL TELECOLLABORATION , 2007 .

[17]  R. Apweiler,et al.  On the Importance of Comprehensible Classification Models for Protein Function Prediction , 2010, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[18]  Dr. Alex A. Freitas Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[19]  L. S. Vygotskiĭ,et al.  Mind in society : the development of higher psychological processes , 1978 .

[20]  Bart Baesens,et al.  An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models , 2011, Decis. Support Syst..

[21]  Ryan Shaun Joazeiro de Baker,et al.  Does the length of time off-task matter? , 2012, LAK.

[22]  N. Law,et al.  Productive Multivocality in the Analysis of Group Interactions , 2013 .

[23]  César Hervás-Martínez,et al.  JCLEC: a Java framework for evolutionary computation , 2007, Soft Comput..

[24]  Sebastián Ventura,et al.  An interpretable classification rule mining algorithm , 2013, Inf. Sci..

[25]  Sean Luke,et al.  Two fast tree-creation algorithms for genetic programming , 2000, IEEE Trans. Evol. Comput..

[26]  L. Vygotsky Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[27]  Sebastián Ventura,et al.  Predicting Student Grades in Learning Management Systems with Multiple Instance Learning Genetic Programming , 2009, EDM.

[28]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[29]  Carolina Ruiz,et al.  The Composition Effect: Conjuntive or Compensatory? An Analysis of Multi-Skill Math Questions in ITS , 2008, EDM.

[30]  David E. Goldberg,et al.  Genetic algorithms and Machine Learning , 1988, Machine Learning.

[31]  Sebastián Ventura,et al.  Predicting students' final performance from participation in on-line discussion forums , 2013, Comput. Educ..

[32]  Cristóbal Romero,et al.  Induction of Classification Rules with Grammar-Based Genetic Programming , 2005 .

[33]  Stefan Hrastinski,et al.  A theory of online learning as online participation , 2009, Comput. Educ..

[34]  A. N. Leont’ev The Problem of Activity in Psychology , 1974 .

[35]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[36]  Erkki Sutinen,et al.  Using data mining for improving web-based course design , 2002, International Conference on Computers in Education, 2002. Proceedings..

[37]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[38]  Carlos Márquez-Vera,et al.  Predicting student failure at school using genetic programming and different data mining approaches with high dimensional and imbalanced data , 2013, Applied Intelligence.

[39]  Sean P. Goggins,et al.  Social Intelligence in Completely Online Groups - Toward Social Prosthetics from Log Data Analysis and Transformation , 2010, 2010 IEEE Second International Conference on Social Computing.

[40]  Peter A. Whigham,et al.  Grammar-based Genetic Programming: a survey , 2010, Genetic Programming and Evolvable Machines.

[41]  N. Galambos,et al.  What Satisfies Students? Mining Student-Opinion Data with Regression and Decision Tree Analysis , 2002 .

[42]  B. Minaei,et al.  Predicting GPA and academic dismissal in LMS using educational data mining: A case mining , 2012, 6th National and 3rd International Conference of E-Learning and E-Teaching.

[43]  Richard Alterman,et al.  Predicting failure: a case study in co-blogging , 2012, LAK.

[44]  Hanan Ayad,et al.  Student success system: risk analytics and data visualization using ensembles of predictive models , 2012, LAK.

[45]  Wilhelmiina Hämäläinen,et al.  Comparison of Machine Learning Methods for Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[46]  Riccardo Poli,et al.  Genetic Programming - Introduction, Applications, Theory and Open Issues , 2012, Handbook of Natural Computing.

[47]  Kevin B. Korb,et al.  Bayesian Artificial Intelligence , 2004, Computer science and data analysis series.

[48]  Zachary A. Pardos,et al.  The Effect of Model Granularity on Student Performance Prediction Using Bayesian Networks , 2007, User Modeling.

[49]  Stephen Fancsali,et al.  Variable construction for predictive and causal modeling of online education data , 2011, LAK.

[50]  Sebastián Ventura,et al.  Predicting Student Grades in Learning Management Systems with Multiple Instance Genetic Programming. , 2009, EDM 2009.

[51]  Mike Sharkey,et al.  Course correction: using analytics to predict course success , 2012, LAK '12.

[52]  Zdenek Zdráhal,et al.  Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment , 2013, LAK '13.

[53]  Sean P. Goggins,et al.  Network Analytic Techniques for Online Chat , 2013 .

[54]  Alaa M. El-Halees,et al.  Mining educational data to improve students' performance: a case study , 2012 .

[55]  Francisco Herrera,et al.  A Survey on the Application of Genetic Programming to Classification , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[56]  Henrik Boström,et al.  Reducing High-Dimensional Data by Principal Component Analysis vs. Random Projection for Nearest Neighbor Classification , 2006, 2006 5th International Conference on Machine Learning and Applications (ICMLA'06).

[57]  Joseph L. Hellerstein,et al.  Recognizing End-User Transactions in Performance Management , 2000, AAAI/IAAI.

[58]  Y. Engeström,et al.  Activity theory and individual and social transformation. , 1999 .

[59]  Marie Bienkowski,et al.  Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief , 2012 .

[60]  CHRISTINE A. HALVERSON,et al.  Activity Theory and Distributed Cognition: Or What Does CSCW Need to DO with Theories? , 2002, Computer Supported Cooperative Work (CSCW).

[61]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[62]  Sotiris B. Kotsiantis,et al.  Predicting students marks in Hellenic Open University , 2005, Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05).

[63]  Luo Si,et al.  Predicting Correctness of Problem Solving from Low-level Log Data in Intelligent Tutoring Systems , 2009, EDM.

[64]  Kelly Blincoe,et al.  Creating a model of the dynamics of socio-technical groups , 2013, User Modeling and User-Adapted Interaction.

[65]  Rebecca Ferguson,et al.  Social Learning Analytics , 2012, J. Educ. Technol. Soc..

[66]  Àngela Nebot,et al.  Identification of fuzzy models to predict students performance in an e-learning environment , 2006 .

[67]  Miguel Ángel Conde González,et al.  Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning , 2014, Comput. Hum. Behav..

[68]  Susan M. Land,et al.  Theoretical Foundations of Learning Environments. , 1999 .

[69]  Tony Gorschek,et al.  Genetic programming for cross-release fault count predictions in large and complex software projects , 2010 .

[70]  Etienne Wenger,et al.  Communities of Practice: Learning, Meaning, and Identity , 1998 .

[71]  Lisa C. Yamagata-Lynch,et al.  Using Activity Theory to Understand the Systemic Tensions Characterizing a Technology-Rich Introductory Astronomy Course , 2002 .

[72]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[73]  Sebastián Ventura,et al.  Solving Classification Problems Using Genetic Programming Algorithms on GPUs , 2010, HAIS.

[74]  Binghui Zheng,et al.  Evolutionary Algorithm for Water Storage Forecasting Response to Climate Change with Small Data Sets: The Wolonghu Wetland, China , 2012 .

[75]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.