A Comparative Study of Modified Principal Component Analysis and Generalized Latent Semantic Analysis Approach to Automated Marking of Theoretical-Based Exams

Examination is a means by which individual’s ability on an acquired skill or knowledge is evaluated. Some examiners useMultiple Choice Questions (MCQ), while others use the theory-based Questions. This study performed a comparative study of two techniques: Modified Principal Component Analysis and Generalized Latent Semantic Analysis used in grading students’ responses in theory-based examination. Softcopy of students’ responses and examiner’s marking scheme were captured in an electronic format of .txt. file. The inherent stopwords and stemming in the .txt document were pre-processed to address morphological variations using standard stopwords list and porters stemmer algorithm, respectively. N-gram terms were derived for each student’s response and the marking schemes (MS) using the vector space model. A Document Term Matrix (DTM) was generated with n-gram terms of MS and students response representing columns and rows, respectively. MPCA and GLSA algorithms were used to reduce the sparseness of the DTM to obtain a vector representation of the students’ answers and the marking scheme. The reduced vector representation of the students’ answers were graded according to the mark assigned to each question in the marking scheme using cosine similarity measure. The developed Automated Theory-Based Marking System (ATBMS) was implemented in Matrix Laboratory 8.1 (R2013b). Performance of MPCA was compared with GLSA to determine the effectiveness of ATBMS on the grading of students’ answers in COM 317 and COM 325 courses, in terms of Pearson correlation coefficient(r) and coefficient of determination (R 2 ).These performance evaluation shows that MPCA is a better feature extraction techniques.