A framework for the computerized assessment of university student essays

This paper presents an approach for automatic grading of essays. Student essays are compared against a model or key essay provided by the teacher. The similarity between a student essay and the model essay is measured by the cosine of their contained angle in an n-dimensional semantic space. The model essay is preprocessed by removing stopwords, extracting keywords, assigning weights to keywords to reflect their importance and finally by linking every keyword to a subject-oriented synonym list. The student essay, by comparison, is preprocessed by removing stopwords and then by extracting keywords. The keywords extracted from the model essay and the keywords extracted from students essays together with weights provided by teacher are used to build feature vectors for teacher and students essays. The obtained grade depends on the similarity between these vectors (calculated by using the cosine formula). A simulator was implemented to test the viability of the proposed approach. It was fed with student essays (at the university level) gathered from database management course over three semesters. The results were very encouraging and the agreement between the auto-grader and human grader was as good as the agreement between human graders.

[1]  Martin Chodorow,et al.  Automated Scoring Using A Hybrid Feature Identification Technique , 1998, ACL.

[2]  David Jackson,et al.  Grading student programs using ASSYST , 1997, SIGCSE '97.

[3]  Peter W. Foltz,et al.  Automated Essay Scoring: Applications to Educational Technology , 1999 .

[4]  Deryle Lonsdale,et al.  Automated Rating of ESL Essays , 2003, HLT-NAACL 2003.

[5]  Athanasios Tsintsifas,et al.  The marking system for CourseMaster , 2002, ITiCSE '02.

[6]  Jan B. Hext,et al.  An automatic grading scheme for simple programming exercises , 1969, Commun. ACM.

[7]  Carolyn Penstein Rosé,et al.  A Hybrid Text Classification Approach for Analysis of Student Essays , 2003, HLT-NAACL 2003.

[8]  Surendra Prasad,et al.  Automatic Evaluation of Students’ Answers using Syntactically Enhanced LSA , 2003, HLT-NAACL 2003.

[9]  Leah S. Larkey,et al.  Automatic essay grading using text categorization techniques , 1998, SIGIR '98.

[10]  urs von Matt.  Kassandra: the automatic grading system , 1994 .

[11]  David D. Lewis,et al.  An evaluation of phrasal and clustered representations on a text categorization task , 1992, SIGIR '92.

[12]  Michael Luck,et al.  Effective electronic marking for on-line assessment , 1998, ITiCSE '98.

[13]  Martin Chodorow,et al.  C-rater: Automated Scoring of Short-Answer Questions , 2003, Comput. Humanit..

[14]  Michael T. Goodrich,et al.  PILOT: an interactive tool for learning and grading , 2000, SIGCSE '00.

[15]  T. Landauer.  Automatic Essay Assessment , 2003 .

[16]  Colin Higgins,et al.  PRAM: prolog automatic marker , 1998, ITiCSE '98.

[17]  Howard R. Mzumara,et al.  On-line Grading of Student Essays: PEG goes on the World Wide Web , 2001 .