AXIS: Generating Explanations at Scale with Learnersourcing and Machine Learning

While explanations may help people learn by providing information about why an answer is correct, many problems on online platforms lack high-quality explanations. This paper presents AXIS (Adaptive eXplanation Improvement System), a system for obtaining explanations. AXIS asks learners to generate, revise, and evaluate explanations as they solve a problem, and then uses machine learning to dynamically determine which explanation to present to a future learner, based on previous learners' collective input. Results from a case study deployment and a randomized experiment demonstrate that AXIS elicits and identifies explanations that learners find helpful. Providing explanations from AXIS also objectively enhanced learning, when compared to the default practice where learners solved problems and received answers without explanations. The rated quality and learning benefit of AXIS explanations did not differ from explanations generated by an experienced instructor.

[1]  Michael S. Bernstein,et al.  Apparition: Crowdsourced User Interfaces that Come to Life as You Sketch Them , 2015, CHI.

[2]  Alexander Renkl,et al.  Learning from Worked-Out-Examples: A Study on Individual Differences , 1997, Cogn. Sci..

[3]  Joseph Jay Williams,et al.  The role of explanation in discovery and generalization: evidence from category learning , 2010, ICLS.

[4]  Zoran Popovic,et al.  Trading Off Scientific Knowledge and User Learning with Multi-Armed Bandits , 2014, EDM.

[5]  Lihong Li,et al.  An Empirical Evaluation of Thompson Sampling , 2011, NIPS.

[6]  Krzysztof Z. Gajos,et al.  Data-driven interaction techniques for improving navigation of educational videos , 2014, UIST.

[7]  Adam J. Berinsky,et al.  Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk , 2012, Political Analysis.

[8]  Joseph E. Beck,et al.  High-Level Student Modeling with Machine Learning , 2000, Intelligent Tutoring Systems.

[9]  Brad A. Myers,et al.  User interface software tools , 1995, TCHI.

[10]  Vincent Aleven,et al.  An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor , 2002, Cogn. Sci..

[11]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[12]  John Langford,et al.  The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information , 2007, NIPS.

[13]  Matthew W. Lewis,et al.  Self-Explonations: How Students Study and Use Examples in Learning to Solve Problems , 1989, Cogn. Sci..

[14]  Juho Kim,et al.  Improving learning with collective learner activity , 2015 .

[15]  Walter S. Lasecki,et al.  Real-time captioning by groups of non-experts , 2012, UIST.

[16]  Leonid Rozenblit,et al.  The misunderstood limits of folk science: an illusion of explanatory depth , 2002, Cogn. Sci..

[17]  Neil T. Heffernan,et al.  The MOOClet Framework: Improving Online Education through Experimentation and Personalization of Modules , 2014 .

[18]  Ron Kohavi,et al.  Responsible editor: R. Bayardo. , 2022 .

[19]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[20]  Pierre-Yves Oudeyer,et al.  Online Optimization of Teaching Sequences with Multi-Armed Bandits , 2014, EDM.

[21]  Arthur C. Graesser,et al.  AutoTutor: an intelligent tutoring system with mixed-initiative dialogue , 2005, IEEE Transactions on Education.

[22]  Mitchell J. Nathan,et al.  Expert Blind Spot : When Content Knowledge Eclipses Pedagogical Content Knowledge , 2001 .

[23]  T. Lombrozo.  The structure and function of explanations , 2006, Trends in Cognitive Sciences.

[24]  Tiffany Barnes,et al.  Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data , 2008, Intelligent Tutoring Systems.

[25]  Krzysztof Z. Gajos,et al.  Learnersourcing Subgoal Labels for How-to Videos , 2015, CSCW.