Not all wizards are from Oz: Iterative design of intelligent learning environments by communication capacity tapering

This paper presents a methodology for the design of intelligent learning environments. We recognise that in the educational technology field, theory development and system-design should be integrated and rely on an iterative process that addresses: (a) the difficulty to elicit precise, concise, and operationalised knowledge from 'experts' and (b) the crucial differences between the communication modalities that experts can relate to, and those that are available to a computer-based system. Inspired by the well-known wizard-of-Oz methodology we discuss the need for characterising and carefully controlling the range of its possible variations. We refer to our approach as 'tapering' of the communication capacity of carefully engineered didactical situations and present its application, and a case study from our work with an exploratory environment. We then discuss the generality of the methodology and pragmatic constraints which can be useful in similar research.

[1]  Judith Good,et al.  CARSS: A Framework for Learner-Centred Design with Children , 2006, Int. J. Artif. Intell. Educ..

[2]  Clayton Lewis,et al.  Designing for usability—key principles and what designers think , 1983, CHI '83.

[3]  Andrea A. diSessa,et al.  Ontological Innovation and the Role of Theory in Design Experiments , 2004 .

[4]  Nicolas Balacheff,et al.  Epistemological domain of validity of microworlds: the case of LOGO and Cabri-Géomètre , 1993, Lessons from Learning.

[5]  Celia Hoyles,et al.  Windows on Mathematical Meanings: Learning Cultures and Computers , 1996 .

[6]  Paul Brna,et al.  User Modeling 2005, 10th International Conference, UM 2005, Edinburgh, Scotland, UK, July 24-29, 2005, Proceedings , 2005, User Modeling.

[7]  Saul Greenberg,et al.  Prototyping an intelligent agent through Wizard of Oz , 1993, INTERCHI.

[8]  James Wilson,et al.  Rapid Prototyping for User Interface Design , 1988 .

[9]  Seymour Papert,et al.  The Cambridge Handbook of the Learning Sciences: Afterword: After How Comes What , 2005 .

[10]  Mark H. Chignell,et al.  A taxonomy of user interface terminology , 1990, SGCH.

[11]  T. Landauer,et al.  Handbook of Human-Computer Interaction , 1997 .

[12]  Albert T. Corbett,et al.  Intelligent Tutoring Systems , 1985, Science.

[13]  Philippe Kruchten,et al.  The Rational Unified Process: An Introduction , 1998 .

[14]  R. Sawyer.  The Cambridge Handbook of the Learning Sciences: Introduction , 2014 .

[15]  Ning Wang,et al.  A Semi-automated Wizard of Oz Interface for Modeling Tutorial Strategies , 2005, User Modeling.

[16]  Mike Sharples,et al.  Socio-cognitive Engineering , 2006 .

[17]  Richard E. Clark,et al.  Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching , 2006 .

[18]  Andrea A. diSessa,et al.  Issues in Component Computing: A Synthetic Review , 2004, Interact. Learn. Environ..

[19]  Arne Jönsson,et al.  Wizard of Oz studies: why and how , 1993, IUI '93.

[20]  G. Brousseau.  Theory of didactical situations in mathematics , 1997 .

[21]  Manolis Mavrikis,et al.  A LEARNING ENVIRONMENT TO SUPPORT MATHEMATICAL GENERALISATION IN THE CLASSROOM , 2009 .

[22]  Celia Hoyles,et al.  Messing Up: Reflections on Introducing Cabri Géomètre , 1994 .

[23]  Michael B. Twidale,et al.  Redressing the balance: the advantages of informal evaluation techniques for Intelligent Learning Environments , 1993 .

[24]  L. Cohen,et al.  Research Methods in Education , 1980 .

[25]  Manolis Mavrikis,et al.  Broadening the sense of ‘dynamic’: a microworld to support students’ mathematical generalisation , 2009 .

[26]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[27]  W. Lewis Johnson,et al.  Iterative Evaluation of a Large-Scale, Intelligent Game for Language Learning , 2005, AIED.

[28]  Manolis Mavrikis,et al.  Diagnosing and acting on student affect: the tutor’s perspective , 2008, User Modeling and User-Adapted Interaction.

[29]  Helen Pain,et al.  Persistent collaboration: a methodology for applied AIED , 1996 .

[30]  Niels Ole Bernsen,et al.  Designing interactive speech systems - from first ideas to user testing , 1998 .

[31]  Niels Pinkwart,et al.  CoChemEx: Supporting Conceptual Chemistry Learning Via Computer-Mediated Collaboration Scripts , 2008, EC-TEL.

[32]  Helmut Horacek,et al.  A Tool for Supporting Progressive Refinement of Wizard-of-Oz Experiments in Natural Language , 2004 .

[33]  Susanne Burger,et al.  Eliciting Natural Speech From Non-Native Users: Collecting Speech Data for LVCSR , 1999 .

[34]  Ning Wang,et al.  Experimental evaluation of polite interaction tactics for pedagogical agents , 2005, IUI.

[35]  Mike Sharples,et al.  Socio-cognitive engineering: A methodology for the design of human-centred technology , 2002, European Journal of Operational Research.

[36]  Barry W. Boehm,et al.  A spiral model of software development and enhancement , 1986, Computer.

[37]  Manolis Mavrikis,et al.  A constructionist approach to mathematical generalisation , 2008 .

[38]  Dennis R. Wixon,et al.  Building a user-derived interface , 1984, CACM.

[39]  Ivana Kruijff-Korbayová,et al.  DiaWOz-II - A Tool for Wizard-of-Oz Experiments in Mathematics , 2006, KI.

[40]  Cristina Conati,et al.  Evaluating a Probabilistic Model of Student Affect , 2004, Intelligent Tutoring Systems.