Learning with intelligent tutors and worked examples: selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum

The main learning activity provided by intelligent tutoring systems is problem solving, although several recent projects investigated the effectiveness of combining problem solving with worked examples. Previous research has shown that learning from examples is an effective learning strategy, especially for novice learners. A worked example provides step-by-step explanations of how a problem is solved. Many studies have compared learning from examples to unsupported problem solving, and suggested presenting worked examples to students in the initial stages of learning, followed by problem solving once students have acquired enough knowledge. This paper presents a study in which we compare a fixed sequence of alternating worked examples and tutored problem solving with a strategy that adapts learning tasks to students’ needs. The adaptive strategy determines the type of the task (a worked example, a faded example or a problem to be solved) based on how much assistance the student received on the previous problem. The results show that students in the adaptive condition learnt significantly more than their peers who were presented with a fixed sequence of worked examples and problem solving. Novices from the adaptive condition learnt faster than novices from the control group, while the advanced students from the adaptive condition learnt more than their peers from the control group.

[1]  Judith Masthoff Design and Evaluation of a Navigation Agent with a Mixed Locus of Control , 2002, Intelligent Tutoring Systems.

[2]  Albert T. Corbett,et al.  A Cognitive Tutor for Genetics Problem Solving: Learning Gains and Student Modeling , 2010 .

[3]  Antonija Mitrovic,et al.  Facilitating Adaptive Tutorial Dialogues in EER-Tutor , 2011, AIED.

[4]  Ann L. Brown,et al.  Preschool children can learn to transfer: Learning to learn and learning from example , 1988, Cognitive Psychology.

[5]  Kenneth R. Koedinger,et al.  When and how often should worked examples be given to students? New results and a summary of the current state of research , 2008 .

[6]  Kurt VanLehn,et al.  The Design of Self-explanation Prompts: The Fit Hypothesis , 2009 .

[7]  Antonija Mitrovic,et al.  Do novices and advanced students benefit differently from worked examples and ITS , 2013 .

[8]  Tamara van Gog,et al.  The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in computer-based learning environments , 2016, Comput. Hum. Behav..

[9]  Antonija Mitrovic,et al.  Examples and Tutored Problems: Adaptive Support Using Assistance Scores , 2015, IJCAI.

[10]  Antonija Mitrovic,et al.  An Intelligent SQL Tutor on the Web , 2003, Int. J. Artif. Intell. Educ..

[11]  Vincent Aleven,et al.  An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor , 2002, Cogn. Sci..

[12]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[13]  Antonija Mitrovic,et al.  Towards Individualized Dialogue Support for Ill-Defined Domains , 2009, Int. J. Artif. Intell. Educ..

[14]  John Hattie,et al.  Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement , 2008 .

[15]  Antonija Mitrovic,et al.  Adaptive Support versus Alternating Worked Examples and Tutored Problems: Which Leads to Better Learning? , 2014, UMAP.

[16]  Slava Kalyuga Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction , 2007 .

[17]  Ryan Shaun Joazeiro de Baker,et al.  Differential Impact of Learning Activities Designed to Support Robust Learning in the Genetics Cognitive Tutor , 2013, AIED.

[18]  Alexander Renkl,et al.  Computer-supported example-based learning: When instructional explanations reduce self-explanations , 2006, Comput. Educ..

[19]  Slava Kalyuga,et al.  When problem solving is superior to studying worked examples. , 2001 .

[20]  Lisa Anthony,et al.  Developing Handwriting-based Intelligent Tutors to Enhance Mathematics Learning , 2008 .

[21]  Birgitta König-Ries,et al.  Progressor: social navigation support through open social student modeling , 2013, New Rev. Hypermedia Multim..

[22]  M. Chi,et al.  The Content of Physics Self-Explanations , 1991 .

[23]  John Sweller,et al.  The worked-example effect using ill-defined problems: Learning to recognise designers' styles , 2009 .

[24]  Fred G. W. C. Paas,et al.  The Efficiency of Instructional Conditions: An Approach to Combine Mental Effort and Performance Measures , 1992 .

[25]  Neil T. Heffernan,et al.  Tutored Problem Solving vs. “Pure” Worked Examples , 2009 .

[26]  Gordon I. McCalla,et al.  Using Planning Techniques in Intelligent Tutoring Systems , 1986, Int. J. Man Mach. Stud..

[27]  N. Webb Peer interaction and learning in small groups , 1989 .

[28]  T. Gog,et al.  Example-Based Learning: Integrating Cognitive and Social-Cognitive Research Perspectives , 2010 .

[29]  John Sweller,et al.  The Effect of Worked Examples When Learning to Write Essays in English Literature , 2013 .

[30]  Peter Brusilovsky,et al.  Evaluation of topic-based adaptation and student modeling in QuizGuide , 2015, User Modeling and User-Adapted Interaction.

[31]  Peter Brusilovsky,et al.  ELM-ART – An Interactive and Intelligent Web-Based Electronic Textbook , 2015, International Journal of Artificial Intelligence in Education.

[32]  Gerhard Weber,et al.  Example Explanation in Learning Environments , 1996, Intelligent Tutoring Systems.

[33]  Alexander Renkl,et al.  Learning how to use a computer-based concept-mapping tool: Self-explaining examples helps , 2009, Comput. Hum. Behav..

[34]  S. Derry,et al.  Learning from Examples: Instructional Principles from the Worked Examples Research , 2000 .

[35]  A. Mitrovic,et al.  EXAMPLES AND TUTORED PROBLEMS: IS ALTERNATING EXAMPLES AND PROBLEMS THE BEST INSTRUCTIONAL STRATEGY? , 2014 .

[36]  J. Sweller The worked example effect and human cognition , 2006 .

[37]  K. Koedinger,et al.  Accounting for Beneficial Effects of Worked Examples in Tutored Problem Solving , 2010 .

[38]  Julita Vassileva,et al.  Reactive Instructional Planning to Support Interacting Teaching Strategies , 1995 .

[39]  Michelene T. H. Chi,et al.  Eliciting Self-Explanations Improves Understanding , 1994, Cogn. Sci..

[40]  F. Paas,et al.  Instructional Efficiency: Revisiting the Original Construct in Educational Research , 2008 .

[41]  Vincent Aleven,et al.  Worked Examples and Tutored Problem Solving: Redundant or Synergistic Forms of Support? , 2009, Top. Cogn. Sci..

[42]  T. Gog,et al.  Effects of worked examples, example-problem, and problem-example pairs on novices learning , 2011 .

[43]  K. Koedinger,et al.  Exploring the Assistance Dilemma in Experiments with Cognitive Tutors , 2007 .

[44]  Bruce M. McLaren,et al.  When Is It Best to Learn with All Worked Examples? , 2011, AIED.

[45]  Fred Paas,et al.  Self-assessment and task selection in learner-controlled instruction: Differences between effective and ineffective learners , 2010, Comput. Educ..

[46]  Demetrios G. Sampson,et al.  Adaptive instructional planning using ontologies , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[47]  Antonija Mitrovic,et al.  Using Learning Curves to Mine Student Models , 2005, User Modeling.

[48]  Jesus G. Boticario,et al.  Educational Recommender Systems and Technologies: Practices and Challenges , 2011 .

[49]  J. Sweller,et al.  The Use of Worked Examples as a Substitute for Problem Solving in Learning Algebra , 1985 .

[50]  Andreas Holzinger,et al.  Efficiency in Learning: Evidence-Based Guidelines to Manage Cognitive Load , 2007, J. Educ. Technol. Soc..

[51]  Tamara van Gog Effects of identical example-problem and problem-example pairs on learning , 2011, Comput. Educ..

[52]  Elena Trichina,et al.  Learning benefits of structural example-based adaptive tutoring systems , 2003, IEEE Trans. Educ..

[53]  Antonija Mitrovic,et al.  Evaluating the effectiveness of feedback in SQL-Tutor , 2000, Proceedings International Workshop on Advanced Learning Technologies. IWALT 2000. Advanced Learning Technology: Design and Development Issues.

[54]  Antonija Mitrovic,et al.  Facilitating deep learning through self-explanation in an open-ended domain , 2006, Int. J. Knowl. Based Intell. Eng. Syst..

[55]  Antonija Mitrovic,et al.  Scaffolding and Fading Problem Selection in SQL-Tutor , 2003 .

[56]  Peter Brusilovsky,et al.  Adaptive navigation support in educational hypermedia: the role of student knowledge level and the case for meta-adaptation , 2003, Br. J. Educ. Technol..

[57]  Gerhard Weber,et al.  Examples and Remindings in a Case-Based Help System , 1994, EWCBR.

[58]  Jeffrey D. Karpicke,et al.  Test-Enhanced Learning , 2006, Psychological science.

[59]  Slava Kalyuga,et al.  Rapid dynamic assessment of expertise to improve the efficiency of adaptive e-learning , 2005 .

[60]  Antonija Mitrovic,et al.  Examples and Tutored Problems: How Can Self-Explanation Make a Difference to Learning? , 2013, AIED.

[61]  Andrew Deeks Web-based Assignments in Structural Analysis , 1999 .

[62]  John Sweller,et al.  Cognitive Load Theory , 2020, Encyclopedia of Education and Information Technologies.

[63]  Antonija Mitrovic,et al.  Revisiting Ill-Definedness and the Consequences for ITSs , 2009, AIED.

[64]  Vincent Aleven,et al.  The worked-example effect: Not an artefact of lousy control conditions , 2009, Comput. Hum. Behav..

[65]  P. Chandler,et al.  Cognitive Load Theory and the Format of Instruction , 1991 .