Evidence-based Approach to Interacting with Open Student Models

Research efforts focused on developing "active reports" are currently underway. Active reports are designed to foster communication among teachers, students, and parents by listening to all stakeholders, using assessment information to guide teaching and learning, and to reconcile potential conflicts. Open student models can handle different views of the student. These views support different kinds of assessment information coming from a variety of sources (e.g. results from summative and formative assessments). As teachers, students, and parents interact with open student models, assessment claims (e.g. students' self-assessment, systems' and teachers' assessment of student knowledge) may evolve from unsupported claims to evidence-based arguments. Paths on this evidentiary argument space are characterized by rich interactions among sources of evidence that may challenge personal claims and improve metacognitive skills. This paper (a) presents an evidence-based approach to interacting with open student models that builds on existing research on open student models, evidentiary arguments and evidence-centered design, (b) illustrates this approach in the context of an assessment-based learning environment called Math Intervention Module (MIM), and (c) reports on a study carried out with Algebra teachers/assessment specialists aimed at evaluating the feasibility of implementing this approach in real settings.

[1]  Valerie J. Shute,et al.  TENSIONS, TRENDS, TOOLS, AND TECHNOLOGIES: TIME FOR AN EDUCATIONAL SEA CHANGE , 2006 .

[2]  H. Pain,et al.  ' Did I say what I think I said , and do you agree with me ? ' : Inspecting and Questioning the Student Model , 1995 .

[3]  Susan Bull,et al.  Helping Learners to Understand Themselves with a Learner Model Open to Students, Peers and Instructors , 2002 .

[4]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[5]  M. Baker A model for negotiation in teaching-learning dialogues , 1994 .

[6]  Judy Kay,et al.  The um toolkit for cooperative user modelling , 2005, User Modeling and User-Adapted Interaction.

[7]  R. Almond,et al.  Focus Article: On the Structure of Educational Assessments , 2003 .

[8]  Valerie J. Shute,et al.  English ABLE , 2007, AIED.

[9]  D. Suthers,et al.  Belvedere: Engaging students in critical discussion of science and public policy issues. , 1995 .

[10]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[11]  Trevor J. M. Bench-Capon Specification and Implementation of Toulmin Dialogue Game , 1999 .

[12]  Antonija Mitrovic,et al.  Supporting Learning by Opening the Student Model , 2002, Intelligent Tutoring Systems.

[13]  J. Greer,et al.  Analysing Student Reflection in The Learning Game , 2003 .

[14]  Michael J. Baker,et al.  Negotiated tutoring : an approach to interaction in intelligent tutoring systems , 1990 .

[15]  Claude Frasson,et al.  Advanced Annotation and External Representation in LKC System , 2002 .

[16]  Paul Brna,et al.  Approximate Modelling of the Multi-dimensional Learner , 2006, Intelligent Tutoring Systems.

[17]  Paul Brna,et al.  Negotiated Collaborative Assessment through Collaborative Student Modelling , 1999 .

[18]  Kikumi K. Tatsuoka,et al.  Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. , 1995 .

[19]  Ana Paiva,et al.  TAGUS — A user and learner modeling workbench , 2005, User Modeling and User-Adapted Interaction.

[20]  Vania Dimitrova,et al.  STyLE-OLM: Interactive Open Learner Modelling , 2003, Int. J. Artif. Intell. Educ..

[21]  Ricardo Conejo,et al.  SIETTE: A Web-Based Tool for Adaptive Testing , 2004, Int. J. Artif. Intell. Educ..

[22]  Rosemary Luckin,et al.  Artificial Intelligence in Education, Building Technology Rich Learning Contexts That Work, Proceedings of the 13th International Conference on Artificial Intelligence in Education, AIED 2007, July 9-13, 2007, Los Angeles, California, USA , 2007, AIED.

[23]  Judy Kay,et al.  Artificial intelligence in education : shaping the future of learning through intelligent technologies , 2003 .

[24]  D. Schum The Evidential Foundations of Probabilistic Reasoning , 1994 .

[25]  Russell G. Almond,et al.  Evaluating ACED: The Impact of Feedback and Adaptivity on Learning , 2007, AIED.

[26]  Robert J. Mislevy,et al.  Accessibility of Tests for Individuals with Disabilities within a Validity Framework. , 2005 .

[27]  Paul Brna,et al.  Opening up the Interpretation Process in an Open Learner Model , 2007, Int. J. Artif. Intell. Educ..

[28]  Jim Greer,et al.  Learning environments based on inspectable student models , 2003 .

[29]  S. Bull,et al.  Reactions to Inspectable Learner Models : Seven Year Olds to University Students , 2005 .

[30]  David Stuart Robertson,et al.  Argument-based applications to knowledge engineering , 2000, The Knowledge Engineering Review.

[31]  Susan Bull Collaborative student modelling in foreign language learning , 1996 .

[32]  Bart Verheij Evaluating Arguments Based on Toulmin’s Scheme , 2005 .

[33]  C. Dwyer,et al.  The Future of Assessment : Shaping Teaching and Learning , 2008 .

[34]  Sunyoung Joung,et al.  Scaffolding collaborative argumentation in asynchronous discussions with message constraints and message labels , 2007, Comput. Educ..

[35]  Jim E. Greer,et al.  Exploring Various Guidance Mechanisms to Support Interaction with Inspectable Learner Models , 2002, Intelligent Tutoring Systems.