On-Line Assessment of Expertise

Abstract : Cognitive science has developed a number of experimental paradigms that appear promising as tools for assessing expertise. However, they typically involve verbal report data and much labor by human analysts. This project will automate some of these paradigms and integrate them into a prototype system for assessment of an individual's progress in achieving expertise in a particular task domain. We have chosen college physics as the task domain. Not only is this task domain rich in conceptual as well as procedural knowledge, but many expert-novice experiments used physics as the task domain, so we can build in the results of these studies, which allows us to concentrate on automating their measurement and analysis. From the students' point of view, the OLAE (On-line Assessment of Expertise) system is a workbench for studying examples and solving problems which occasionally interrupts to ask for an explanation of one's action. OLAE presents standard physics examples and problems, as well as qualitative physics problems, problem classification tasks, and other tasks. The student will attend lectures and read the prose of the textbook offline, but all the rest of the student's work will be done on OLAE.