A Comparison of Student Evaluation Algorithms in AutoTutor

Intelligent tutoring systems (ITSs) require adept student modeling mechanisms in order to adapt their pedagogical strategies to their users. One key element of any student model is its evaluation algorithm for student responses. AutoTutor, a natural language intelligent tutoring system developed at the University of Memphis (Graesser, Chipman et al., 2005), uses a pair of evaluation algorithms based on the statistics of language. AutoTutor combines Latent Semantic Analysis (LSA) (Landauer, Foltz, & Laham, 1998) with a string matching algorithm in which each word is weighted relative to its inverse word frequency (IWFO). This hybrid approach aims to provide a good evaluation of student input without requiring considerable knowledge engineering. Past analyses have suggested this hybrid algorithm is likely to have good agreement with expert ratings (Graesser et al., 2007). We believe a hybrid model will outperform either LSA or IWFO alone.