Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection - Wizards Go to School in the Elementary Maths Classroom

The work described in this paper investigates the potential of Automatic Speech Recognition (ASR) to support young children's exploration and reflection as they are working with interactive learning environments. We describe a unique ecologically valid Wizard-of-Oz (WoZ) study in a classroom equipped with computers, two of which were set up to allow human facilitators (wizards) to listen to students thinking-aloud while having access to their interaction with the environment. The wizards provided support using a script and following an iterative methodology that limited on purpose their communication capacity in order to simulate the actual system. Our results indicate that the feedback received from the wizards did serve its function i.e. it helped modify students' behaviour in that they did think-aloud significantly more than in past interactions and rephrased their language to employ mathematical terminology. Additional results from student perception questionnaires show that overall students find the system suggestions helpful, not repetitive and understandable. Most also enjoy thinking aloud to the computer but, as expected, some find the feedback cognitively overloading, indicating that more work is needed on how to design the interaction tipping the balance towards facilitating post-task reflection.

[1]  J. Flavell,et al.  The development of children's knowledge about inner speech. , 1997, Child development.

[2]  A. Schoenfeld.  Cognitive Science and Mathematics Education , 1987 .

[3]  Michelene T. H. Chi,et al.  The Cambridge Handbook of Multimedia Learning: The Self-Explanation Principle in Multimedia Learning , 2005 .

[4]  Helene Alpert Furani.  MISCONCEIVING O R MISNAMING?: SOME IMPLICATIONS OF TODDLERS' SYMBOLIZING FOR MATHEMATICS EDUCATION 1 , 2000 .

[5]  Ruth Wylie,et al.  The Self-Explanation Principle in Multimedia Learning , 2014 .

[6]  S. Stein.  What's All the Fuss About? , 1996 .

[7]  Margit Askeland,et al.  Sound-based strategy training in multiplication , 2012 .

[8]  Manolis Mavrikis,et al.  Not all wizards are from Oz: Iterative design of intelligent learning environments by communication capacity tapering , 2010, Comput. Educ..

[9]  R. Glaser.  Advances in Instructional Psychology , 1978 .

[10]  Manolis Mavrikis,et al.  Diagnosing and acting on student affect: the tutor’s perspective , 2008, User Modeling and User-Adapted Interaction.

[11]  Helene Alpert Furani.  MISCONCEIVING OR MISNAMING ? : SOME IMPLICATIONS OF TODDLERS ’ SYMBOLIZING FOR MATHEMATICS EDUCATION , 2022 .

[12]  Manolis Mavrikis,et al.  Design requirements, student perception indicators and validation metrics for intelligent exploratory learning environments , 2013, Personal and Ubiquitous Computing.

[13]  Richard Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[14]  Nia Dowell,et al.  Does it really matter whether students' contributions are spoken versus typed in an intelligent tutoring system with natural language? , 2011, Journal of experimental psychology. Applied.

[15]  H. Freudenthal.  Major problems of mathematics education , 1981 .

[16]  John F. Canny,et al.  The Future of Human-Computer Interaction , 2006, ACM Queue.

[17]  N. Mercer,et al.  Teaching Children How to Use Language to Solve Maths Problems , 2006 .

[18]  Jack Mostow,et al.  Evaluating tutors that listen: an overview of project LISTEN , 2001 .

[19]  Lars Schmidt-Thieme,et al.  Feature Analysis for Affect Recognition Supporting Task Sequencing in Adaptive Intelligent Tutoring Systems , 2014, EC-TEL.

[20]  J. Read,et al.  Endurability, Engagement and Expectations: Measuring Children’s Fun , 2002 .

[21]  Marjorie Siegel,et al.  Using Transactional Reading Strategies To Support Sense-Making and Discussion in Mathematics Classrooms: An Exploratory Study. , 1998 .

[22]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .