Educational Data Mining and Learning Analytics

In recent years, two communities have grown around a joint interest on how big data can be exploited to benefit education and the science of learning: Educational Data Mining and Learning Analytics. This article discusses the relationship between these two communities, and the key methods and approaches of educational data mining. The article discusses how these methods emerged in the early days of research in this area, which methods have seen particular interest in the EDM and learning analytics communities, and how this has changed as the field matures and has moved to making significant contributions to both educational research and practice.

[1]  G. Tutz,et al.  An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. , 2009, Psychological methods.

[2]  Linda Corrin,et al.  Visualizing patterns of student engagement and performance in MOOCs , 2014, LAK.

[3]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[4]  Mark Wilson,et al.  A framework for item response models , 2004 .

[5]  Kenneth R. Koedinger,et al.  Is Over Practice Necessary? - Improving Learning Efficiency with the Cognitive Tutor through Educational Data Mining , 2007, AIED.

[6]  W. F. Punch,et al.  Predicting student performance: an application of data mining methods with an educational Web-based system , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[7]  Mykola Pechenizkiy,et al.  Handbook of Educational Data Mining , 2010 .

[8]  Taylor Martin,et al.  Learning Programming with IPRO: The Effects of a Mobile, Social Programming Environment. , 2013 .

[9]  Neil T. Heffernan,et al.  Addressing the assessment challenge with an online system that tutors as it assesses , 2009, User Modeling and User-Adapted Interaction.

[10]  R. Nisbett,et al.  Immediate and delayed transfer of training effects in statistical reasoning. , 1991, Journal of experimental psychology. General.

[11]  Ryan Shaun Joazeiro de Baker,et al.  Using Text Replay Tagging to Produce Detectors of Systematic Experimentation Behavior Patterns , 2010, EDM.

[12]  Ryan Shaun Joazeiro de Baker,et al.  Leveraging machine-learned detectors of systematic inquiry behavior to estimate and predict transfer of inquiry skill , 2011, User Modeling and User-Adapted Interaction.

[13]  Ryan Shaun Joazeiro de Baker,et al.  Modeling and understanding students' off-task behavior in intelligent tutoring systems , 2007, CHI.

[14]  Ryan Shaun Joazeiro de Baker,et al.  Reengineering the Feature Distillation Process: A case study in detection of Gaming the System , 2014, EDM.

[15]  Zlatko J. Kovacic,et al.  Early Prediction of Student Success: Mining Students Enrolment Data , 2010 .

[16]  Doug Clow Data wranglers: human interpreters to help close the feedback loop , 2014, LAK '14.

[17]  Charles Anderson,et al.  The end of theory: The data deluge makes the scientific method obsolete , 2008 .

[18]  Vanessa P. Dennen,et al.  From Message Posting to Learning Dialogues: Factors affecting learner participation in asynchronous discussion , 2005 .

[19]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[20]  Vincent Aleven,et al.  Sensor-free automated detection of affect in a Cognitive Tutor for Algebra , 2012, EDM.

[21]  C. Spearman The proof and measurement of association between two things. By C. Spearman, 1904. , 1987, The American journal of psychology.

[22]  John P. Campbell,et al.  Academic Analytics: A New Tool for a New Era. , 2007 .

[23]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[24]  Jessica Lin,et al.  Finding Motifs in Time Series , 2002, KDD 2002.

[25]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[26]  David Nadler Prata,et al.  Dialogue Analysis in Collaborative Learning , 2012 .

[27]  Vania Dimitrova,et al.  Visualising student tracking data to support instructors in web-based distance education , 2004, WWW Alt. '04.

[28]  Stephen Fancsali Variable Construction and Causal Discovery for Cognitive Tutor Log Data: Initial Results , 2012, EDM.

[29]  Judy Kay,et al.  Analysing Frequent Sequential Patterns of Collaborative Learning Activity Around an Interactive Tabletop. Nominee for Best Paper Award , 2010, EDM.

[30]  Robert A. Muenchen,et al.  The Popularity of Data Analysis Software , 2013 .

[31]  Albert T. Corbett,et al.  A Bayes Net Toolkit for Student Modeling in Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[32]  R. Almond,et al.  A BRIEF INTRODUCTION TO EVIDENCE-CENTERED DESIGN , 2003 .

[33]  Vivienne L. Ming,et al.  Predicting student outcomes from unstructured data , 2012, UMAP Workshops.

[34]  D. J. Summers,et al.  Charm physics at Fermilab E791 , 1992 .

[35]  Eitel J. M. Lauría,et al.  Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative , 2014, J. Learn. Anal..

[36]  Kalina Yacef,et al.  Educational Data Mining: a Case Study , 2005, AIED.

[37]  Gautam Biswas,et al.  Identifying Learning Behaviors by Contextualizing Differential Sequence Mining with Action Features and Performance Evolution , 2012, EDM.

[38]  Ryan Shaun Joazeiro de Baker,et al.  WTF? detecting students who are conducting inquiry without thinking fastidiously , 2012, UMAP.

[39]  Judy Kay,et al.  Clustering and Sequential Pattern Mining of Online Collaborative Learning Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[40]  Arnon Hershkovitz,et al.  The Impact of Off-task and Gaming Behaviors on Learning: Immediate or Aggregate? , 2009, AIED.

[41]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[42]  Mykola Pechenizkiy,et al.  Predicting Students Drop Out: A Case Study , 2009, EDM.

[43]  David J. Hand,et al.  Data Mining: Statistics and More? , 1998 .

[44]  Osmar R. Zaïane,et al.  Analyzing Participation of Students in Online Courses Using Social Network Analysis Techniques , 2011, EDM.

[45]  Gregory K. W. K. Chung,et al.  Identifying Key Features of Student Performance in Educational Video Games and Simulations through Cluster Analysis , 2012, EDM 2012.

[46]  S. Messick The Interplay of Evidence and Consequences in the Validation of Performance Assessments , 1994 .

[47]  H. P. Bahrick,et al.  Maintenance of Foreign Language Vocabulary and the Spacing Effect , 1993 .

[48]  Marta E. Zorrilla,et al.  Social Network Analysis and Data Mining: An Application to the E-Learning Context , 2013, ICCCI.

[49]  Ryan Shaun Joazeiro de Baker,et al.  Adapting to When Students Game an Intelligent Tutoring System , 2006, Intelligent Tutoring Systems.

[50]  Andrew Olney,et al.  Mining Collaborative Patterns in Tutorial Dialogues , 2010, EDM 2010.

[51]  Gautam Biswas,et al.  Identifying Students' Characteristic Learning Behaviors in an Intelligent Tutoring System Fostering Self-Regulated Learning , 2012, EDM.

[52]  Ryan Shaun Joazeiro de Baker,et al.  Detecting Student Misuse of Intelligent Tutoring Systems , 2004, Intelligent Tutoring Systems.

[53]  J. Greeno THE SITUATIVITY OF KNOWING, LEARNING, AND RESEARCH , 1998 .

[54]  Taylor Martin,et al.  Learning Fractions by Splitting: Using Learning Analytics to Illuminate the Development of Mathematical Understanding , 2015 .

[55]  Chirag Patel,et al.  Exploring a Joint Model of Conventional and Online Learning Systems , 2006 .

[56]  J. Manyika Big data: The next frontier for innovation, competition, and productivity , 2011 .

[57]  Vincent Aleven,et al.  Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra. , 2012, EDM 2012.

[58]  Dror Ben-Naim,et al.  A User-Driven and Data-Driven Approach for Supporting Teachers in Reflection and Adaptation of Adaptive Tutorials , 2009, EDM.

[59]  R. Schmidt,et al.  New Conceptualizations of Practice: Common Principles in Three Paradigms Suggest New Concepts for Training , 1992 .

[60]  Sébastien George,et al.  TrAVis to enhance online tutoring and learning activities: Real-time visualization of students tracking data , 2011, Interact. Technol. Smart Educ..

[61]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[62]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[63]  Kasia Muldner,et al.  An analysis of students’ gaming behaviors in an intelligent tutoring system: predictors and impacts , 2011, User Modeling and User-Adapted Interaction.

[64]  Sebastián Ventura,et al.  An architecture for making recommendations to courseware authors using association rule mining and collaborative filtering , 2009, User Modeling and User-Adapted Interaction.

[65]  Bruce M. McLaren,et al.  Supporting Collaborative Learning and E-Discussions Using Artificial Intelligence Techniques , 2010, Int. J. Artif. Intell. Educ..

[66]  Tristan Nixon,et al.  A Method for Finding Prerequisites Within a Curriculum , 2011, EDM.

[67]  George Siemens,et al.  Learning analytics and educational data mining: towards communication and collaboration , 2012, LAK.

[68]  Alex Paramythis,et al.  Activity sequence modelling and dynamic clustering for personalized e-learning , 2011, User Modeling and User-Adapted Interaction.

[69]  Neil T. Heffernan,et al.  Predicting College Enrollment from Student Interaction with an Intelligent Tutoring System in Middle School , 2013, EDM.

[70]  E. Mandinach,et al.  The role of cognitive engagement in classroom learning and motivation , 1983 .

[71]  Tiffany Barnes,et al.  The Q-matrix Method: Mining Student Response Data for Knowledge , 2005 .

[72]  Michel C. Desmarais,et al.  Improving matrix factorization techniques of student test data with partial order constraints , 2012, UMAP.

[73]  John R. Anderson,et al.  ACT-R: A Theory of Higher Level Cognition and Its Relation to Visual Attention , 1997, Hum. Comput. Interact..

[74]  Joel D. Martin,et al.  Student assessment using Bayesian nets , 1995, Int. J. Hum. Comput. Stud..

[75]  Marcelo Worsley,et al.  Towards the development of multimodal action based assessment , 2013, LAK '13.

[76]  Gurmak Singh,et al.  Implementing eLearning Programmes for Higher Education: A Review of the Literature , 2004, J. Inf. Technol. Educ..

[77]  Jonathan P. Rowe,et al.  When Off-Task is On-Task: The Affective Role of Off-Task Behavior in Narrative-Centered Learning Environments , 2011, AIED.

[78]  Joseph E. Beck,et al.  Exploring User Data From a Game-like Math Tutor: A Case Study in Causal Modeling , 2011, EDM.

[79]  Valerie J. Shute,et al.  SMART: Student modeling approach for responsive tutoring , 1995, User Modeling and User-Adapted Interaction.

[80]  Cristina Conati,et al.  Combining Unsupervised and Supervised Classification to Build User Models for Exploratory , 2009, EDM 2009.

[81]  Gautam Biswas,et al.  Using Hidden Markov Models to Characterize Student Behaviors in Learning-by-Teaching Environments , 2008, Intelligent Tutoring Systems.

[82]  Lei Qu,et al.  Classifying Learner Engagement through Integration of Multiple Data Sources , 2006, AAAI.

[83]  Ryan Shaun Joazeiro de Baker,et al.  An Analysis of the Differences in the Frequency of Students' Disengagement in Urban, Rural, and Suburban High Schools , 2010, EDM.

[84]  Chris Piech,et al.  Deconstructing disengagement: analyzing learner subpopulations in massive open online courses , 2013, LAK '13.

[85]  Ryan Shaun Joazeiro de Baker,et al.  Developing a generalizable detector of when students game the system , 2008, User Modeling and User-Adapted Interaction.

[86]  Cristina Conati,et al.  Incorporating an Affective Behavior Model into an Educational Game , 2009, FLAIRS Conference.

[87]  Mitsuru Ikeda,et al.  Proceedings of the 8th international conference on Intelligent Tutoring Systems , 2006 .

[88]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[89]  Arthur C. Graesser,et al.  Automatic detection of learner’s affect from conversational cues , 2008, User Modeling and User-Adapted Interaction.

[90]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[91]  Neil T. Heffernan,et al.  Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[92]  Beverly Park Woolf,et al.  Identifying High-Level Student Behavior Using Sequence-based Motif Discovery , 2010, EDM.

[93]  Mathieu d'Aquin,et al.  Interpreting data mining results with linked data for learning analytics: motivation, case study and directions , 2013, LAK '13.

[94]  Kikumi K. Tatsuoka,et al.  Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. , 1995 .

[95]  Antonija Mitrovic,et al.  Evaluation of a Constraint-Based Tutor for a Database Language , 1999 .

[96]  R Scheines,et al.  The TETRAD Project: Constraint Based Aids to Causal Model Specification. , 1998, Multivariate behavioral research.

[97]  Russell G. Almond,et al.  Bayesian Networks in Educational Assessment , 2015 .

[98]  Vincent Aleven,et al.  Intelligent Tutoring Goes To School in the Big City , 1997 .

[99]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[100]  John R Anderson,et al.  Using a model to compute the optimal schedule of practice. , 2008, Journal of experimental psychology. Applied.

[101]  Caroline Haythornthwaite,et al.  Exploring Multiplexity: Social Network Structures in a Computer-Supported Distance Learning Class , 2001, Inf. Soc..

[102]  Albert T. Corbett,et al.  A Cognitive Tutor for Genetics Problem Solving: Learning Gains and Student Modeling , 2010 .

[103]  Taylor Martin,et al.  Using Learning Analytics to Understand the Learning Pathways of Novice Programmers , 2013 .

[104]  Ryan Shaun Joazeiro de Baker,et al.  Automatically Detecting a Student's Preparation for Future Learning: Help Use is Key , 2011, EDM.

[105]  Ryan Shaun Joazeiro de Baker,et al.  The Relationship between Carelessness and Affect in a Cognitive Tutor , 2011, ACII.

[106]  Dieter Fensel,et al.  Knowledge Engineering: Principles and Methods , 1998, Data Knowl. Eng..

[107]  Daniel L. Schwartz,et al.  Inventing to Prepare for Future Learning: The Hidden Efficiency of Encouraging Original Student Production in Statistics Instruction , 2004 .

[108]  Taylor Martin,et al.  Nanogenetic learning analytics: illuminating student learning pathways in an online fraction game , 2013, LAK '13.

[109]  Kenneth R. Koedinger,et al.  Automated Student Model Improvement , 2012, EDM.

[110]  Edward A. Feigenbaum,et al.  The fifth generation - artificial intelligence and Japan's computer challenge to the world , 1991 .

[111]  Dragan Gamberger,et al.  Combining Unsupervised and Supervised Machine Learning , 2001, AIME.

[112]  Rebecca Ferguson,et al.  Visualizing social learning ties by type and topic: rationale and concept demonstrator , 2013, LAK '13.

[113]  Valerie J. Shute,et al.  A Large-Scale Evaluation of an Intelligent Discovery World: Smithtown , 1990, Interact. Learn. Environ..

[114]  Judy Kay,et al.  The Big Five and Visualisations of Team Work Activity , 2006, Intelligent Tutoring Systems.

[115]  Kenneth R. Koedinger,et al.  A Response Time Model For Bottom-Out Hints as Worked Examples , 2008, EDM.

[116]  Jeff Grann,et al.  Competency map: visualizing student learning to promote student success , 2014, LAK.

[117]  I. E. Allen,et al.  Staying the course: online education in the United States, 2008 , 2008 .

[118]  John O'Malley,et al.  Students Perceptions of Distance Learning, Online Learning and the Traditional Classroom , 1999 .

[119]  Mathieu d'Aquin Putting Linked Data to Use in a Large Higher-Education Organisation , 2012, ILD@ESWC.

[120]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[121]  Daniel D. Suthers,et al.  A unified framework for multi-level analysis of distributed learning , 2011, LAK.

[122]  Johan Jeuring,et al.  University students' achievement goals and help-seeking strategies in an intelligent tutoring system , 2014, Comput. Educ..

[123]  Vincent Aleven,et al.  Learner Differences in Hint Processing , 2012, EDM.

[124]  Lei Qu,et al.  Mathematics motivation and achievement as predictors of high school students' guessing and help-seeking with instructional software , 2008, J. Comput. Assist. Learn..

[125]  Jody Clarke,et al.  Predicting Successful Inquiry Learning in a Virtual Performance Assessment for Science , 2013, UMAP.

[126]  K. Tatsuoka RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .

[127]  Zachary A. Pardos,et al.  The sum is greater than the parts: ensembling models of student knowledge in educational software , 2012, SKDD.

[128]  Mladen A. Vouk,et al.  Experimental Analysis of the Q-Matrix Method in Knowledge Discovery , 2005, ISMIS.

[129]  Robert J. Mislevy,et al.  An Application of Exploratory Data Analysis in the Development of Game-Based Assessments , 2015 .

[130]  Judy Kay,et al.  Exploiting Readily Available Web Data for Scrutable Student Models , 2005, AIED.

[131]  Beverly Park Woolf,et al.  Inferring learning and attitudes from a Bayesian Network of log file data , 2005, AIED.

[132]  Mimi Recker,et al.  Understanding Teacher Users of a Digital Library Service: A Clustering Approach , 2011, EDM 2011.

[133]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[134]  Neil T. Heffernan,et al.  How to Construct More Accurate Student Models: Comparing and Optimizing Knowledge Tracing and Performance Factor Analysis , 2011, Int. J. Artif. Intell. Educ..

[135]  Ryan Shaun Joazeiro de Baker,et al.  Cross-System Transfer of Machine Learned and Knowledge Engineered Models of Gaming the System , 2015, UMAP.

[136]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[137]  Jing Zhang,et al.  EDUCATIONAL DATA MINING , 2016 .

[138]  Michel C. Desmarais,et al.  Item to Skills Mapping: Deriving a Conjunctive Q-matrix from Data , 2012, ITS.

[139]  Peter Maurer,et al.  The Cambridge Handbook of the Learning Sciences , 2022 .

[140]  Peter Norvig,et al.  The Unreasonable Effectiveness of Data , 2009, IEEE Intelligent Systems.

[141]  Vincent Aleven,et al.  Argument graph classification with Genetic Programming and C4.5 , 2008, EDM.

[142]  D. Klahr,et al.  All other things being equal: acquisition and transfer of the control of variables strategy. , 1999, Child development.

[143]  Ryan Shaun Joazeiro de Baker,et al.  Towards Predicting Future Transfer of Learning , 2011, AIED.

[144]  Shane Dawson,et al.  SNAPP: a bird's-eye view of temporal participant interaction , 2011, LAK.

[145]  F. Collins,et al.  The Human Genome Project: Lessons from Large-Scale Biology , 2003, Science.

[146]  Rebecca Ferguson,et al.  Social Learning Analytics , 2012, J. Educ. Technol. Soc..

[147]  Neil T. Heffernan,et al.  Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[148]  Sebastián Ventura,et al.  Educational data mining: A survey from 1995 to 2005 , 2007, Expert Syst. Appl..

[149]  Diane Jass Ketelhut,et al.  Automatic Grading of Scientific Inquiry , 2012, BEA@NAACL-HLT.

[150]  Carolyn Penstein Rosé,et al.  Challenging Assumptions: using sliding window visualizations to reveal time-based irregularities in CSCL processes , 2012, ICLS.

[151]  Rajeev Motwani,et al.  Dynamic itemset counting and implication rules for market basket data , 1997, SIGMOD '97.

[152]  Alex Pentland,et al.  Sensing and modeling human networks using the sociometer , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[153]  S. Natasha Beretvas,et al.  Comparing Multidimensional and Unidimensional Proficiency Classifications: Multidimensional IRT as a Diagnostic Aid , 2003 .

[154]  Albert T. Corbett,et al.  The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning , 2012, Cogn. Sci..

[155]  Krzysztof Z. Gajos,et al.  Understanding in-video dropouts and interaction peaks inonline lecture videos , 2014, L@S.

[156]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[157]  V. Aleven,et al.  Help Seeking and Help Design in Interactive Learning Environments , 2003 .

[158]  Judy Kay,et al.  An Interactive Teacher's Dashboard for Monitoring Groups in a Multi-tabletop Learning Environment , 2012, ITS.

[159]  Alex J. Bowers Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of Students: Grades, Data Driven Decision Making, Dropping out and Hierarchical Cluster Analysis. , 2010 .

[160]  Gautam Biswas,et al.  Developing Learning by Teaching Environments That Support Self-Regulated Learning , 2004, Intelligent Tutoring Systems.

[161]  Vincent Aleven,et al.  Can Help Seeking Be Tutored? Searching for the Secret Sauce of Metacognitive Tutoring , 2007, AIED.

[162]  Albert T. Corbett,et al.  Does Help Help? Introducing the Bayesian Evaluation and Assessment Methodology , 2008, Intelligent Tutoring Systems.

[163]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[164]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[165]  Ramakrishnan Srikant,et al.  Mining Sequential Patterns: Generalizations and Performance Improvements , 1996, EDBT.

[166]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[167]  Kalina Yacef,et al.  Interestingness Measures for Associations Rules in Educational Data , 2008, EDM.

[168]  Bertrand Meyer,et al.  EMPIRICAL STUDY OF NOVICE ERRORS AND ERROR PATHS IN OBJECT-ORIENTED PROGRAMMING , 2006 .

[169]  Neil T. Heffernan,et al.  Towards Live Informing and Automatic Analyzing of Student Learning: Reporting in ASSISTment System , 2007 .

[170]  Paul Kline,et al.  An easy guide to factor analysis , 1993 .

[171]  Jody Clarke,et al.  Towards Identifying Students' Causal Reasoning Using Machine Learning , 2013, AIED.

[172]  Neil T. Heffernan,et al.  Population validity for educational data mining models: A case study in affect detection , 2014, Br. J. Educ. Technol..

[173]  Carolyn Penstein Rosé,et al.  Using Machine Learning Techniques to Analyze and Support Mediation of Student E-Discussions , 2007, AIED.

[174]  Kenneth R. Koedinger,et al.  A Data Repository for the EDM Community: The PSLC DataShop , 2010 .

[175]  Richard Scheines,et al.  Searching for Variables and Models to Investigate Mediators of Learning from Multiple Representations , 2012, EDM.

[176]  John D. Storey The positive false discovery rate: a Bayesian interpretation and the q-value , 2003 .

[177]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[178]  Arthur C. Graesser,et al.  Automatic Discovery of Speech Act Categories in Educational Games , 2012, EDM.

[179]  Judy Kay,et al.  Analyzing Collaborative Interactions with Data Mining Methods for the Benefit of Learning , 2011 .

[180]  Beverly Park Woolf,et al.  Repairing Disengagement With Non-Invasive Interventions , 2007, AIED.

[181]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[182]  Kimberly E. Arnold Signals: Applying Academic Analytics. , 2010 .

[183]  Michel C. Desmarais Conditions for Effectively Deriving a Q-Matrix from Data with Non-negative Matrix Factorization. Best Paper Award , 2011, EDM.

[184]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[185]  Ryan Shaun Joazeiro de Baker,et al.  Contextual Slip and Prediction of Student Performance after Use of an Intelligent Tutor , 2010, UMAP.

[186]  Ryan Shaun Joazeiro de Baker,et al.  Development of a Workbench to Address the Educational Data Mining Bottleneck , 2012, EDM.

[187]  Zachary A. Pardos,et al.  Affective states and state tests: investigating how affect throughout the school year predicts end of year learning outcomes , 2013, LAK '13.

[188]  David E. Pritchard,et al.  Correlating skill and improvement in 2 MOOCs with a student's time on tasks , 2014, L@S.

[189]  Shane Dawson,et al.  A Study of the Relationship between Student Social Networks and Sense of Community , 2008, J. Educ. Technol. Soc..

[190]  Brian C. Nelson,et al.  Presence and Middle School Students' Participation in a Virtual Game Environment to Assess Science Inquiry , 2012, J. Educ. Technol. Soc..

[191]  Albert T. Corbett,et al.  Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments , 2008 .

[192]  Steven F. Quigley,et al.  Computer-based formative assessment to promote reflection and learner autonomy , 2006 .

[193]  Zachary A. Pardos,et al.  MOOCdb: Developing Data Standards for MOOC Data Science , 2013 .

[194]  R. Sawyer The Cambridge Handbook of the Learning Sciences: Introduction , 2014 .

[195]  Janice D. Gobert,et al.  From Log Files to Assessment Metrics: Measuring Students' Science Inquiry Skills Using Educational Data Mining , 2013, Journal of the Learning Sciences.

[196]  I. E. Allen,et al.  Changing Course: Ten Years of Tracking Online Education in the United States. , 2013 .

[197]  Vincent Aleven,et al.  Toward Meta-cognitive Tutoring: A Model of Help Seeking with a Cognitive Tutor , 2006, Int. J. Artif. Intell. Educ..

[198]  Arnon Hershkovitz,et al.  Developing a Log-based Motivation Measuring Tool , 2008, EDM.

[199]  Luo Si,et al.  Automatic Detection of Off-Task Behaviors in Intelligent Tutoring Systems with Machine Learning Techniques , 2010, IEEE Transactions on Learning Technologies.

[200]  Sebastián Ventura,et al.  A Survey on Pre-Processing Educational Data , 2014 .

[201]  Lara M. Triona,et al.  Point and Click or Grab and Heft: Comparing the Influence of Physical and Virtual Instructional Materials on Elementary School Students' Ability to Design Experiments , 2003 .

[202]  Mike Sharkey,et al.  Academic analytics landscape at the University of Phoenix , 2011, LAK.

[203]  Anthony Chow,et al.  College students' perceived threat and preference for seeking help in traditional, distributed, and distance learning environments , 2007, Comput. Educ..

[204]  Mark Warschauer,et al.  Predicting MOOC performance with Week 1 Behavior , 2014, EDM.

[205]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[206]  Ryan Shaun Joazeiro de Baker,et al.  Improving construct validity yields better models of systematic inquiry, even with less information , 2012, UMAP.

[207]  Ryan Shaun Joazeiro de Baker,et al.  Detecting Carelessness through Contextual Estimation of Slip Probabilities among Students Using an Intelligent Tutor for Mathematics , 2011, AIED.

[208]  Carolyn Penstein Rosé,et al.  Towards Academically Productive Talk Supported by Conversational Agents , 2012, ITS.

[209]  Vincent Aleven,et al.  Educational Software Features that Encourage and Discourage "Gaming the System" , 2009, AIED.