A Crowdsourcing Approach to Collecting Tutorial Videos -- Toward Personalized Learning-at-Scale

We investigated the feasibility of crowdsourcing full- fledged tutorial videos from ordinary people on the Web on how to solve math problems related to logarithms. This kind of approach (a form of learnersourcing [9, 11]) to efficiently collecting tutorial videos and other learning resources could be useful for realizing personalized learning-at-scale, whereby students receive specific learning resources -- drawn from a large and diverse set -- that are tailored to their individual and time-varying needs. Results of our study, in which we collected 399 videos from 66 unique "teachers" on Mechanical Turk, suggest that (1) approximately 100 videos -- over 80% of which are mathematically fully correct -- can be crowdsourced per week for $5/video; (2) the average learning gains (posttest minus pretest score) associated with watching the videos was stat. sig. higher than for a control video (0.105 versus 0.045); and (3) the average learning gains (0.1416) from watching the best tested crowdsourced videos was comparable to the learning gains (0.1506) from watching a popular Khan Academy video on logarithms.

[1]  Valentina Paredes,et al.  A teacher like me or a student like me? Role model versus teacher bias effect , 2014 .

[2]  Julie L. Booth,et al.  Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples , 2013 .

[3]  Beverly Park Woolf,et al.  Affect-aware tutors: recognising and responding to student affect , 2009, Int. J. Learn. Technol..

[4]  Krzysztof Z. Gajos,et al.  Crowdsourcing step-by-step information extraction to enhance existing how-to videos , 2014, CHI.

[5]  Tom Murray,et al.  An Overview of Intelligent Tutoring System Authoring Tools: Updated Analysis of the State of the Art , 2003 .

[6]  Zoran Popovic,et al.  Crowdsourcing Accurate and Creative Word Problems and Hints , 2016, HCOMP.

[7]  Richard G. Baraniuk,et al.  A Contextual Bandits Framework for Personalized Learning Action Selection , 2016, EDM.

[8]  Hans Anand Pant,et al.  Second language learners' performance in mathematics: Disentangling the effects of academic language features , 2013 .

[9]  Krzysztof Z. Gajos,et al.  Learnersourcing subgoal labeling to support learning from how-to videos , 2013, CHI Extended Abstracts.

[10]  T. Dee.  A Teacher Like Me: Does Race, Ethnicity, or Gender Matter? , 2005 .

[11]  Daniel F. McCaffrey,et al.  Effectiveness of Cognitive Tutor Algebra I at Scale , 2014 .

[12]  Neil T. Heffernan,et al.  AXIS: Generating Explanations at Scale with Learnersourcing and Machine Learning , 2016, L@S.

[13]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Lessons Learned , 2005, Int. J. Artif. Intell. Educ..

[14]  Gwo-Jen Hwang,et al.  A Heuristic Algorithm for planning personalized learning paths for context-aware ubiquitous learning , 2010, Comput. Educ..

[15]  A. Ziv,et al.  Teaching and learning with humor: Experiment and replication. , 1988 .

[16]  Chih-Ming Chen,et al.  Intelligent web-based learning system with personalized learning path guidance , 2008, Comput. Educ..

[17]  Sumit Gulwani,et al.  Personalized Mathematical Word Problem Generation , 2015, IJCAI.

[18]  K. Koedinger,et al.  Exploring the Assistance Dilemma in Experiments with Cognitive Tutors , 2007 .

[19]  Tom Routen,et al.  Intelligent Tutoring Systems , 1996, Lecture Notes in Computer Science.

[20]  Selen Türkay,et al.  The effects of whiteboard animations on retention and subjective experiences when learning advanced physics topics , 2016, Comput. Educ..

[21]  Jürgen Baumert,et al.  Students' and mathematics teachers' perceptions of teacher enthusiasm and instruction , 2008 .

[22]  Krzysztof Z. Gajos,et al.  Understanding in-video dropouts and interaction peaks inonline lecture videos , 2014, L@S.

[23]  Janet Wiles,et al.  Characterizing the temporal dynamics of student-teacher discourse , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[24]  Neil T. Heffernan,et al.  The Future of Adaptive Learning: Does the Crowd Hold the Key? , 2016, International Journal of Artificial Intelligence in Education.

[25]  S. Graf,et al.  Adaptive and Intelligent Web-Based Educational Systems , 2009 .

[26]  Tiffany Barnes,et al.  Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data , 2008, Intelligent Tutoring Systems.

[27]  Vincent Aleven,et al.  Creating a Corpus of Targeted Learning Resources with a Web-Based Open Authoring Tool , 2009, IEEE Transactions on Learning Technologies.

[28]  W. Marsden.  I and J , 2012 .

[29]  Jonathan Meer,et al.  The Impact of Teacher–Student Gender Matches: Random Assignment Evidence from South Korea , 2015 .

[30]  Neil T. Heffernan,et al.  The MOOClet Framework: Improving Online Education through Experimentation and Personalization of Modules , 2014 .