Which User Interactions Predict Levels of Expertise in Work-Integrated Learning?

Predicting knowledge levels from user's implicit interactions with an adaptive system is a difficult task, particularly in learning systems that are used in the context of daily work tasks. We have collected interactions of six persons working with the adaptive work-integrated learning system APOSDLE over a period of two months to find out whether naturally occurring interactions with the system can be used to predict their level of expertise. One set of interactions is based on the tasks they performed, the other on a number of additional Knowledge Indicating Events KIE. We find that the addition of KIE significantly improves the prediction as compared to using tasks only. Both approaches are superior to a model that uses only the frequencies of events.

[1]  Julita Vassileva,et al.  Adaptive Hypertext and Hypermedia , 1998, Springer Netherlands.

[2]  Martha C. Polson,et al.  Foundations of intelligent tutoring systems , 1988 .

[3]  Steven M. Downing,et al.  Handbook of test development , 2006 .

[4]  Stefanie N. Lindstaedt,et al.  Providing Varying Degrees of Guidance for Work-Integrated Learning , 2010, EC-TEL.

[5]  Albert T. Corbett,et al.  Cognitive Tutor: Applied research in mathematics education , 2007, Psychonomic bulletin & review.

[6]  Carla Limongelli,et al.  Adaptive Learning with the LS-Plan System: A Field Evaluation , 2009, IEEE Transactions on Learning Technologies.

[7]  Peter Brusilovsky,et al.  ELM-ART: An Intelligent Tutoring System on World Wide Web , 1996, Intelligent Tutoring Systems.

[8]  Stefanie N. Lindstaedt,et al.  Seeing what the system thinks you know: visualizing evidence in an open learner model , 2012, LAK '12.

[9]  Natalia Stash,et al.  The Design of AHA! , 2006, HYPERTEXT '06.

[10]  Peter Brusilovsky,et al.  KnowledgeTree: a distributed architecture for adaptive e-learning , 2004, WWW Alt. '04.

[11]  Frank Linton,et al.  Recommender Systems for Learning: Building User and Expert Models through Long-Term Observation of Application Use , 2000, User Modeling and User-Adapted Interaction.

[12]  Barry R. Nathan,et al.  A comparison of validation criteria: Objective versus subjective performance measures and self- versus supervisor ratings. , 2006 .

[13]  Mark Levene,et al.  Web dynamics : adapting to change in content, size, topology and use , 2004 .

[14]  Judy Kay,et al.  Personis: A Server for User Models , 2002, AH.

[15]  Tobias Ley,et al.  A methodology for eliciting, modelling, and evaluating expert knowledge for an adaptive work-integrated learning system , 2010, Int. J. Hum. Comput. Stud..

[16]  Peter Brusilovsky,et al.  User Models for Adaptive Hypermedia and Adaptive Educational Systems , 2007, The Adaptive Web.

[17]  Stefanie N. Lindstaedt,et al.  Modeling competencies for supporting work-integrated learning in knowledge work , 2008, J. Knowl. Manag..

[18]  Francesco Ricci,et al.  User Modeling, Adaptation, and Personalization , 2014, Lecture Notes in Computer Science.

[19]  Walid Maalej,et al.  Potentials and challenges of recommendation systems for software development , 2008, RSSE '08.

[20]  Peter Brusilovsky,et al.  Domain, task, and user models for an adaptive hypermedia performance support system , 2002, IUI '02.

[21]  Cristina Conati,et al.  Using Bayesian Networks to Manage Uncertainty in Student Modeling , 2002, User Modeling and User-Adapted Interaction.

[22]  Stefanie N. Lindstaedt,et al.  Getting to Know Your User - Unobtrusive User Model Maintenance within Work-Integrated Learning Environments , 2009, EC-TEL.

[23]  Richard Lepsinger,et al.  The Art and Science of 360 Degree Feedback , 1997 .

[24]  Natalia Stash,et al.  The design of AHA! , 2006, HYPERTEXT '06.

[25]  Ralf Muellerbuchhof,et al.  Vergleich subjektiver und objektiver Messverfahren für die Bestimmung von Methodenkompetenz - am Beispiel der Kompetenzmessung bei technischem Fachpersonal , 2004 .

[26]  Fridolin Wild,et al.  Using Latent-Semantic Analysis and Network Analysis for Monitoring Conceptual Development , 2011, J. Lang. Technol. Comput. Linguistics.

[27]  John Schaubroeck,et al.  A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. , 1988 .

[28]  John R. Anderson,et al.  Cognitive Modeling and Intelligent Tutoring , 1990, Artif. Intell..

[29]  Dietrich Albert,et al.  Individualized Skill Assessment in Digital Learning Games: Basic Definitions and Mathematical Formalism , 2011, IEEE Transactions on Learning Technologies.

[30]  Michel C. Desmarais,et al.  A review of recent advances in learner and skill modeling in intelligent learning environments , 2012, User Modeling and User-Adapted Interaction.

[31]  Allen I. Huffcutt,et al.  Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. , 1997 .

[32]  Dietrich Albert,et al.  Competence-based Knowledge Structures for Personalised Learning , 2005, ELeGI Conference.

[33]  Antonija Mitrovic,et al.  Fifteen years of constraint-based tutors: what we have achieved and where we are going , 2011, User Modeling and User-Adapted Interaction.

[34]  Carlo Strapparava,et al.  Adaptive Hypermedia and Adaptive Web-Based Systems, 5th International Conference, AH 2008, Hannover, Germany, July 29 - August 1, 2008. Proceedings , 2008, AH.

[35]  Alfred Kobsa,et al.  The Adaptive Web, Methods and Strategies of Web Personalization , 2007, The Adaptive Web.

[36]  Tobias Ley,et al.  Scaffolding Self-directed Learning with Personalized Learning Goal Recommendations , 2010, UMAP.

[37]  Lora Aroyo,et al.  Adaptive Web-Based Educational Hypermedia , 2004, Web Dynamics.

[38]  Ulrike Cress,et al.  Learning in the Synergy of Multiple Disciplines, 4th European Conference on Technology Enhanced Learning, EC-TEL 2009, Nice, France, September 29 - October 2, 2009, Proceedings , 2009, EC-TEL.

[39]  Stefanie N. Lindstaedt,et al.  Sustaining TEL: From Innovation to Learning and Practice , 2010, Lecture Notes in Computer Science.