Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief

The authors are grateful for the deliberations of our technical working group (TWG) of academic experts in educational data mining and learning analytics. These experts provided constructive guidance and comments for this issue brief. The TWG comprised Ryan S. In data mining and data analytics, tools and techniques once confined to research laboratories are being adopted by forward-looking industries to generate business intelligence for improving decision making. Higher education institutions are beginning to use analytics for improving the services they provide and for increasing student grades and retention. The U.S. Department of Education's National Education Technology Plan, as one part of its model for 21st-century learning powered by technology, envisions ways of using data from online learning systems to improve instruction. With analytics and data mining experiments in education starting to proliferate, sorting out fact from fiction and identifying research possibilities and practical applications are not easy. This issue brief is intended to help policymakers and administrators understand how analytics and data mining have been—and can be—applied for educational improvement. At present, educational data mining tends to focus on developing new tools for discovering patterns in data. These patterns are generally about the microconcepts involved in learning: one-digit multiplication, subtraction with carries, and so on. Learning analytics—at least as it is currently contrasted with data mining—focuses on applying tools and techniques at larger scales, such as in courses and at schools and postsecondary institutions. But both disciplines work with patterns and prediction: If we can discern the pattern in the data and make sense of what is happening, we can predict what should come next and take the appropriate action. Educational data mining and learning analytics are used to research and build models in several areas that can influence online learning systems. One area is user modeling, which encompasses what a learner knows, what a learner's behavior and motivation are, what the user experience is like, and how satisfied users are with online learning. At the simplest level, analytics can detect when a student in an online course is going astray and nudge him or her on to a course correction. At the most complex, they hold promise of detecting boredom from patterns of key clicks and redirecting the student's attention. Because these data are gathered in real time, there is a real possibility of continuous improvement via multiple feedback loops that operate at different time scales—immediate to the student …

[1]  M. Lovett,et al.  The Open Learning Initiative: Measuring the Effectiveness of the OLI Statistics Course in Accelerating Student Learning. , 2008 .

[2]  George Siemens,et al.  Penetrating the fog: analytics in learning and education , 2014 .

[3]  Alfred Kobsa,et al.  Privacy-enhanced personalization , 2006, FLAIRS.

[4]  Neil T. Heffernan,et al.  Addressing the assessment challenge with an online system that tutors as it assesses , 2009, User Modeling and User-Adapted Interaction.

[5]  Tim Coley Defining IT's Role in Mission-Critical Retention Initiatives. , 2010 .

[6]  Cristina Conati,et al.  A Framework for Capturing Distinguishing User Interaction Behaviors in Novel Interfaces , 2011, EDM.

[7]  Eitel J. M. Lauría,et al.  Mining Sakai to Measure Student Performance : Opportunities and Challenges in Academic Analytics ( Research in Progress ) , 2011 .

[8]  Margaret Hilton Protecting Student Records and Facilitating Education Research: A Workshop Summary. , 2008 .

[9]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[10]  KöckMirjam,et al.  Activity sequence modelling and dynamic clustering for personalized e-learning , 2011 .

[11]  William E. Brown,et al.  Improving the Feedback Cycle to Improve Learning in Introductory Biology Using the Digital Dashboard , 2006 .

[12]  Tanya Elias,et al.  Learning Analytics: Definitions, Processes and Potential , 2011 .

[13]  Alfred Kobsa,et al.  Privacy-enhanced personalization , 2007, CACM.

[14]  Dragan Gasevic,et al.  Open Learning Analytics: an integrated modularized platform , 2011 .

[15]  Alan Levine,et al.  The 2010 Horizon Report. , 2010 .

[16]  Shane Dawson,et al.  Social networks adapting pedagogical practice: SNAPP , 2009 .

[17]  Harold F. O'Neil What Works in Distance Learning: Guidelines , 2005 .

[18]  Jeffrey C. Wayman,et al.  Involving Teachers in Data-Driven Decision Making: Using Computer Data Systems to Support Teacher Inquiry and Reflection , 2005 .

[19]  Larry Johnson,et al.  The 2011 Horizon Report. , 2011 .

[20]  Paulo Blikstein,et al.  Using learning analytics to assess students' behavior in open-ended programming tasks , 2011, LAK.

[21]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Lessons Learned , 2005, Int. J. Artif. Intell. Educ..

[22]  S. Dawson,et al.  Harnessing ICT potential: The adoption and analysis of ICT systems for enhancing the student learning experience , 2010 .

[23]  B. Means,et al.  Use of Education Data at the Local Level: From Accountability to Instructional Improvement. , 2010 .

[24]  Antonija Mitrovic,et al.  Evaluating and improving adaptive educational systems with learning curves , 2011, User Modeling and User-Adapted Interaction.

[25]  Neil T. Heffernan,et al.  A Quasi-Experimental Evaluation of An On-Line Formative Assessment and Tutoring System , 2010 .

[26]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[27]  Cristina Conati,et al.  Combining Unsupervised and Supervised Classification to Build User Models for Exploratory , 2009, EDM 2009.

[28]  Barbara Means,et al.  Teaching advanced skills to at-risk students : views from research and practice , 1991 .

[29]  Ryan Shaun Joazeiro de Baker,et al.  Automatically Detecting a Student's Preparation for Future Learning: Help Use is Key , 2011, EDM.

[30]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[31]  Candace Thille,et al.  Assessment and Instruction: Two Sides of the Same Coin , 2008 .

[32]  Gerhard Fischer,et al.  User Modeling in Human–Computer Interaction , 2001, User Modeling and User-Adapted Interaction.

[33]  J. Manyika Big data: The next frontier for innovation, competition, and productivity , 2011 .

[34]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[35]  Alfred Kobsa User Modeling and User-Adapted Interaction , 2005, User Modeling and User-Adapted Interaction.

[36]  Martin Wattenberg,et al.  Harry Potter and the Meat-Filled Freezer: A Case Study of Spontaneous Usage  of Visualization Tools , 2008, Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008).

[37]  Alfred Kobsa,et al.  User modeling in dialog systems: Potentials and hazards , 1990, AI & SOCIETY.

[38]  E. Mandinach,et al.  Using Student Achievement Data to Support Instructional Decision Making. IES Practice Guide. NCEE 2009-4067. , 2009 .

[39]  Andrew Harrison,et al.  Timed Report Measures Learning: Game-Based Embedded Assessment , 2012 .

[40]  Albert T. Corbett,et al.  Cognitive Tutor: Applied research in mathematics education , 2007, Psychonomic bulletin & review.

[41]  Kalina Yacef,et al.  Measuring Correlation of Strong Symmetric Association Rules in Educational Data , 2010 .

[42]  Ryan Shaun Joazeiro de Baker,et al.  Generalizing Detection of Gaming the System Across a Tutoring Curriculum , 2006, Intelligent Tutoring Systems.

[43]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[44]  Jesus Boticario,et al.  A Data Mining Approach to Reveal Representative Collaboration Indicators in Open Collaboration Frameworks , 2009, EDM.

[45]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[46]  Gautam Biswas,et al.  Mining Student Behavior Models in Learning-by-Teaching Environments , 2008, EDM.

[47]  Kimberly E. Arnold Signals: Applying Academic Analytics. , 2010 .

[48]  Kinshuk,et al.  Dynamic Student Modelling of Learning Styles for Advanced Adaptivity in Learning Management Systems , 2013, Int. J. Inf. Syst. Soc. Chang..

[49]  Kenneth R. Koedinger,et al.  A Data Repository for the EDM Community: The PSLC DataShop , 2010 .

[50]  Sebastián Ventura,et al.  Data mining in education , 2013, WIREs Data Mining Knowl. Discov..

[51]  Christopher Krügel,et al.  Abusing Social Networks for Automated User Profiling , 2010, RAID.

[52]  Jack Mostow,et al.  How Who Should Practice: Using Learning Decomposition to Evaluate the Efficacy of Different Types of Practice for Different Types of Students , 2008, Intelligent Tutoring Systems.

[53]  George Siemens,et al.  Learning analytics and educational data mining: towards communication and collaboration , 2012, LAK.

[54]  Alex Paramythis,et al.  Activity sequence modelling and dynamic clustering for personalized e-learning , 2011, User Modeling and User-Adapted Interaction.