Exploring Participative Learner Modelling and Its Effects on Learner Behaviour

The educational benefits of involving learners as active players in the learner modelling process have been an important motivation for research on this form of learner modelling, henceforth referred to as participative learner modelling. Such benefits, conceived as the promotion of learners’ reflection on and awareness of their own knowledge, have in most cases been asserted on the grounds of system design and supported only by anecdotal evidence. This dissertation explores the issue of whether participative learner modelling actually promotes learners’ reflection and awareness. It does so by firstly interpreting ‘reflection’ and ‘awareness’ in light of “classical” theories of human cognitive architecture, skill acquisition and meta-cognition, in order to infer changes in learner abilities (and therefore behaviour) amenable to empirical corroboration. The occurrence of such changes is then tested for an implementation of a paradigmatic form of participative learner modelling: allowing learners to inspect and modify their learner models. The domain of application centres on the sensorimotor skill of controlling a pole on a cart and represents a novel type of domain for participative learner modelling. Special attention is paid to evaluating the method developed for constructing learner models and the form of presenting them to learners: the former is based on a method known as behavioural cloning for acquiring expert knowledge by means of machine learning; the latter deals with the modularity of the learner models and the modality and interactivity of their presentation. The outcome of this research suggests that participative learner modelling may increase the abilities of learners to report accurately their problem-solving knowledge and to carry out novel tasks in the same domain—the sort of behavioural changes expected from increased learners’ awareness and reflection. More importantly perhaps, the research suggests a viable methodology for examining the educational benefits of participative learner modelling. It also exemplifies the difficulties that such endeavours will face.

[1]  Susan Bull.  Individualized Recommendations for Learning Strategy Use , 2000, Intelligent Tutoring Systems.

[2]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 1996, Machine Learning.

[3]  Pierre Dillenbourg,et al.  A Framework for Learner Modelling , 1992, Interact. Learn. Environ..

[4]  Pierre Dillenbourg.  The computer as a constructorium: Tools for observing one's own learning , 1992 .

[5]  Allan Collins,et al.  THE COMPUTER AS A TOOL FOR LEARNING THROUGH REFLECTION , 1986 .

[6]  Bert Bredeweg,et al.  Student Modelling: The Key to Individualized Knowledge-Based Instruction , 2010, NATO ASI Series.

[7]  Luis A. Pineda,et al.  A Model for Multimodal Representation and Inference , 1999, Visual Representations and Interpretations.

[8]  M. Vince.  The intermittency of control movements and the psychological refractory period. , 1948, The British journal of psychology. General section.

[9]  P. Cheng.  Restructuring versus automaticity: Alternative accounts of skill acquisition. , 1985 .

[10]  K. VanLehn,et al.  Cognitive skill acquisition. , 1996, Annual review of psychology.

[11]  Frederick Hayes-Roth,et al.  Rule-based systems , 1985, CACM.

[12]  Paul M. Fitts,et al.  Perceptual-Motor Skill Learning1 , 1964 .

[13]  Bryant J. Cratty.  Teaching Motor Skills , 1973 .

[14]  Judy Kay,et al.  Learner Know Thyself: Student Models to Give Learner Control and Responsibility , 1997 .

[15]  Michael B. Twidale,et al.  Improving error diagnosis using intermediate representations , 1991 .

[16]  Julita Vassileva,et al.  A Multi-Agent Design of a Peer- Help Environment , 1999 .

[17]  G. McCalla.  The Fragmentation of Culture , Learning , Teaching and Technology : Implications for the Artificial Intelligence in Education Research Agenda in 2010 , 2000 .

[18]  Sholom M. Weiss,et al.  Predictive data mining - a practical guide , 1997 .

[19]  Stellan Ohlsson,et al.  Artificial Instruction. A Method for Relating Learning Theory to Instructional Design. , 1992 .

[20]  Daniele Nardi,et al.  Reasoning about Student Knowledge and Reasoning , 1991, IJCAI.

[21]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[22]  John R. Anderson.  The Architecture of Cognition , 1983 .

[23]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[24]  John R. Anderson,et al.  Mapping eye movements to cognitive processes , 1999 .

[25]  Sara Jones,et al.  Animated Diagrams: An Investigation into the Cognitive Effects of Using Animation to Illustrate Dynamic Processes , 2000, Diagrams.

[26]  J. Self.  Dormobile : a vehicle for metacognition , 1995 .

[27]  Donald Michie,et al.  Knowledge, Learning and Machine Intelligence , 1993 .

[28]  Ashwin Srinivasan,et al.  Numerical Reasoning with an ILP System Capable of Lazy Evaluation and Customised Search , 1999, J. Log. Program..

[29]  Stellan Ohlsson,et al.  A Machine Learning Approach to Student Modeling. , 1984 .

[30]  Judy Kay,et al.  The scrutable user model and beyond , 1999 .

[31]  Helen Pain,et al.  Understandable Learner Models for a Sensorimotor Control Task , 2000, Intelligent Tutoring Systems.

[32]  Richard Lowe.  Animation of Diagrams: An Aid to Learning? , 2000, Diagrams.

[33]  Irving John Good,et al.  The Estimation of Probabilities: An Essay on Modern Bayesian Methods , 1965 .

[34]  Vincent Aleven,et al.  Limitations of Student Control: Do Students Know When They Need Help? , 2000, Intelligent Tutoring Systems.

[35]  G. Cumming,et al.  Learner Models in Collaborative Intelligent Educational Systems , 1991 .

[36]  David H. Jonassen,et al.  Handbook of Individual Differences, Learning, and Instruction , 1993 .

[37]  Vania Dimitrov,et al.  The Interactive Maintenance of Open Learner Models , 1999 .

[38]  John Self,et al.  The Role of Student Models in Learning Environments (Special Issue on Intelligent CAI and Hypermedia) , 1994 .

[39]  Etienne Wenger,et al.  Artificial Intelligence and Tutoring Systems , 1987 .

[40]  Donald Michie,et al.  Learning concepts from data , 1998 .

[41]  F. Weinert,et al.  Metacognition, motivation, and understanding , 1987 .

[42]  Susan Bull.  Collaborative student modelling in foreign language learning , 1996 .

[43]  D. Broadbent,et al.  Interactive tasks and the implicit‐explicit distinction , 1988 .

[44]  J. Flavell.  Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry. , 1979 .

[45]  Judy Kay.  Accretion Representation for Scrutable Student Modeling , 2000, Intelligent Tutoring Systems.

[46]  Benjamin C. Kuo,et al.  AUTOMATIC CONTROL SYSTEMS , 1962, Universum:Technical sciences.

[47]  F. Akhras.  Modelling Learning as a Process , 1997 .

[48]  Ronald Fagin,et al.  Reasoning about knowledge , 1995 .

[49]  Edwina L. Rissland,et al.  Cognitive Science: An Introduction , 1987 .

[50]  Stephanie D. Teasley.  Constructing a Joint Problem Space: The Computer as a Tool for Sharing Knowledge , 1993 .

[51]  Susanne P. Lajoie,et al.  Artificial Intelligence in Education: Open Learning Environments: New Computational Technologies to Support Learning , 1999 .

[52]  Michie Jh.  Structure of interactive decision skill in novices and experts. , 2000 .

[53]  Mark H. Lee.  On Models, Modelling and the Distinctive Nature of Model-Based Reasoning , 1999, AI Commun..

[54]  John E. Laird,et al.  Human-Level AI's Killer Application: Interactive Computer Games , 2000, AI Mag..

[55]  A. Brown.  Metacognition, executive control, self-regulation, and other more mysterious mechanisms , 1987 .

[56]  Susan Bull,et al.  See Yourself Write: A Simple Student Model to Make Students Think , 1997 .

[57]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[58]  W. Lewis Johnson,et al.  Understanding and Debugging Novice Programs , 1990, Artif. Intell..

[59]  Z. Dienes,et al.  A theory of implicit and explicit knowledge , 1999, Behavioral and Brain Sciences.

[60]  Brian Everitt,et al.  Cluster analysis , 1974 .

[61]  Jim E. Greer,et al.  Inspecting and Visualizing Distributed Bayesian Student Models , 2000, Intelligent Tutoring Systems.

[62]  John E. Laird,et al.  Learning Hierarchical Performance Knowledge by Observation , 1999, ICML.

[63]  D. Broadbent,et al.  On the Relationship between Task Performance and Associated Verbalizable Knowledge , 1984 .

[64]  Judy Kay,et al.  Studying long-term system use , 1995, CACM.

[65]  Terry Winograd,et al.  Understanding natural language , 1974 .

[66]  Peter A. Frensch,et al.  The Role of Information Reduction in Skill Acquisition , 1996, Cognitive Psychology.

[67]  Riichiro Mizoguchi,et al.  Artificial Intelligence in Education: Knowledge and Media in Learning Systems , 1997 .

[68]  Stellan Ohlsson,et al.  Some principles of intelligent tutoring , 1986 .

[69]  John Seely Brown,et al.  Diagnostic Models for Procedural Bugs in Basic Mathematical Skills , 1978, Cogn. Sci..

[70]  M. Baker.  A model for negotiation in teaching-learning dialogues , 1994 .

[71]  G. Ryle,et al.  心的概念 = The concept of mind , 1962 .

[72]  Rui Camacho,et al.  Inducing Models of human Control Skills , 1998, ECML.

[73]  A. Newell.  Unified Theories of Cognition , 1990 .

[74]  Derek H. Sleeman.  Inferring (Mal) Rules from Pupil's Protocols , 1982, ECAI.

[75]  Paul Brna,et al.  Supporting the use of external representation in problem solving: the need for flexible learning environments , 1995 .

[76]  K. A. Ericsson,et al.  Expert and exceptional performance: evidence of maximal adaptation to task constraints. , 1996, Annual review of psychology.

[77]  John A. Self,et al.  Formal Approaches to Student Modelling , 1994 .

[78]  Robert B. Ammons,et al.  Motor Skills , 1951 .

[79]  Allan Collins,et al.  Misconceptions in student's understanding , 1979 .

[80]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[81]  Yoneo Yano,et al.  Learner Models for Supporting Awareness and Collaboration in a CSCL Environment , 1996, Intelligent Tutoring Systems.

[82]  Sunil Vadera,et al.  A collaborative approach to user modeling with in a multi-functional architecture , 1999 .

[83]  Jaime R. Carbonell,et al.  AI in CAI : An artificial intelligence approach to computer-assisted instruction , 1970 .

[84]  Geoffrey I. Webb,et al.  Comparative evaluation of alternative induction engines for Feature Based Modelling , 1997 .

[85]  Raymund Sison,et al.  Student Modeling and Machine Learning , 1998 .

[86]  Donald Michie,et al.  Cognitive models from subcognitive skills , 1990 .

[87]  John R. Anderson,et al.  The Geometry Tutor , 1985, IJCAI.

[88]  Paul Brna,et al.  Maintaining a Jointly Constructed Student Model , 2000, AIMSA.

[89]  William Bechtel,et al.  Connectionism and the mind: An introduction to parallel processing in networks , 1991 .

[90]  Hugh Coolican.  Research Methods and Statistics in Psychology , 1993 .

[91]  J. Flavell.  Speculations about the nature and development of metacognition , 1987 .

[92]  Rui Camacho,et al.  Building symbolic representations of intuitive real-time skills from performance data , 1994, Machine Intelligence 13.

[93]  Jim E. Greer,et al.  Visualization of Bayesian Learner Models , 1999 .

[94]  Nicholas J. Pioch,et al.  Verbal Coaching During a Real-Time Task , 1998, Intelligent Tutoring Systems.

[95]  L. Schauble,et al.  Beyond Modularity: A Developmental Perspective on Cognitive Science. , 1994 .

[96]  H. Grice.  Logic and conversation , 1975 .

[97]  John Self.  Bypassing the intractable problem of student modelling , 1988 .

[98]  Ivan Bratko,et al.  Reconstructing Human Skill with Machine Learning , 1994, ECAI.

[99]  Geoffrey I. Webb,et al.  A Comparison of First-Order and Zeroth-Order Induction for Input-Output Agent Modelling , 1997 .

[100]  Simon Kasif,et al.  A System for Induction of Oblique Decision Trees , 1994, J. Artif. Intell. Res..

[101]  H. Pain,et al.  ' Did I say what I think I said , and do you agree with me ? ' : Inspecting and Questioning the Student Model , 1995 .

[102]  Pierre Dillenbourg,et al.  People Power: A Human-Computer Collaborative Learning System , 1992, Intelligent Tutoring Systems.

[103]  Tom Conlon.  Alternatives to rules for knowledge-based modelling , 1999 .

[104]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[105]  John Self,et al.  The defining characteristics of intelligent tutoring systems research: ITSs care, precisely , 1998 .

[106]  Jon Oberlander,et al.  A cognitive theory of graphical and linguistic reasoning: logic and implementation. Cognitive Science , 1995 .

[107]  Roger M. Kerr,et al.  Interactive Reactive Scheduling by Human Learning and Machine Induction , 1994 .

[108]  John A. Self.  The application of machine learning to student modelling , 1986 .

[109]  Susan M. Land,et al.  The foundations and assumptions of technology-enhanced student-centered learning environments , 1997 .

[110]  John R. Anderson,et al.  Broader Bandwidth in Student Modeling: What if ITS were "Eye"TS? , 2000, Intelligent Tutoring Systems.

[111]  Julita Vassileva,et al.  Openness and Disclosure in Multi-Agent Learner Models , 1999 .

[112]  Ron Kohavi,et al.  Data Mining Using MLC a Machine Learning Library in C++ , 1996, Int. J. Artif. Intell. Tools.

[113]  Yvonne Barnard,et al.  Self-explanations: do we get them from our students , 1996 .

[114]  Terry Winograd,et al.  FRAME REPRESENTATIONS AND THE DECLARATIVE/PROCEDURAL CONTROVERSY , 1975 .

[115]  E. Eastwood,et al.  Chairman's address: Control & Automation Division. Control theory and the engineer , 1968 .

[116]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[117]  Allen and Rosenbloom Paul S. Newell,et al.  Mechanisms of Skill Acquisition and the Law of Practice , 1993 .

[118]  J. Beck,et al.  Cooperative Student Models , 1997 .

[119]  Helen Pain,et al.  Modelling of novices control skills with machine learning , 1999 .

[120]  John R. Anderson,et al.  Dynamic Student Modelling in an Intelligent Tutor for LISP Programming , 1985, IJCAI.

[121]  Paul Brna,et al.  Negotiated Collaborative Assessment through Collaborative Student Modelling , 1999 .

[122]  Susan Bull,et al.  Raising Learner Awareness of Language Learning Strategies in Situations of Limited Resources , 2001, Interact. Learn. Environ..

[123]  A. Makarovic.  A qualitative way of solving the pole balancing problem , 1991 .

[124]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[125]  Rodney Cotterill,et al.  No ghost in the machine , 1989 .

[126]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[127]  Paul P. Maglio,et al.  The computational basis of interactive skill , 1996 .

[128]  John Seely Brown,et al.  An Investigation of Computer Coaching for Informal Learning Activities. , 1978 .

[129]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .