A system for formative assessment and monitoring of students' progress

Abstract Assessment plays a central role in any educational process as a way of evaluating the students' knowledge on the concepts associated with learning objectives. The assessment of free-text answers is a process that, besides being very costly in terms of time spent by teachers, may lead to inequities due to the difficulty in applying the same evaluation criteria to all answers. This paper describes a system composed by several modules whose main goal is to work as a formative assessment tool for students and to help teachers creating and assessing exams as well monitoring students' progress. The system automatically creates training exams for students to practice based on questions from previous exams and assists teachers in the creation of evaluation exams with various kinds of information about students' performance. The system automatically assesses training exams to give automatic feedback to students. The correction of free-text answers is based on the syntactic and semantic similarity between the student answers and various reference answers, thus going beyond the simple lexical matching. For this, several pre-processing tasks are performed in order to reduce each answer to its more manageable canonical form. Besides the syntactic and semantic similarity between answers, the way the teacher evaluates the answers is also acquired. To accomplish that, the assessment is done using sub scores defined by the teacher concerning parts of the answer or its subgoals. The system has been trained and tested on exams manually graded by History teachers. There is a good correlation between the evaluation of the instructors and the evaluation performed by our system.