Automated Short Answer Scoring using Weighted Cosine Coefficient

Scoring short answer question (consisting of at most 100 words) requires a special treatment, particularly on the term weighting process. With such limitation, it is therefore infeasible to have term weighting with term frequency model because the frequency of term appearance is very rare. This paper proposes a new method, named Weighted Cosine Coefficient. The experiment result shows that Weighted Cosine Coefficient method can improve the correlation value on Automated Short Answer Scoring (ASAS) system compared to that of Jaccard Coefficient, Dice Coefficient, Cosine Coefficient, Weighted Jaccard Coefficient, and Weighted Dice Coefficient. The average improvement of the correlation value on the tested data set is 0.56.

[1]  Wenyin Liu,et al.  Term Weighting Schemes for Question Categorization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Wael Hassan Gomaa,et al.  Short Answer Grading Using String Similarity And Corpus-Based Similarity , 2012 .

[3]  Feng Zhao.  The Algorithm Analyses and Design about the Subjective Test Online Basing on the DOM Tree , 2008, 2008 International Conference on Computer Science and Software Engineering.

[4]  V. Thada,et al.  Comparison of Jaccard, Dice, Cosine Similarity Coefficient To Find Best Fitness Value for Web Retrieved Documents Using Genetic Algorithm , 2013 .

[5]  V. Senthil Kumaran,et al.  Towards an automated system for short-answer assessment using ontology mapping , 2015, Int. Arab. J. e Technol..

[6]  Kinshuk,et al.  Auto-Assessor: Computerized Assessment System for Marking Student's Short-Answers Automatically , 2011, 2011 IEEE International Conference on Technology for Education.

[7]  Amitkumar Manekar,et al.  Online Examination with short text matching , 2014, 2014 IEEE Global Conference on Wireless Computing & Networking (GCWCN).

[8]  Fakhroddin Noorbehbahani,et al.  The automatic assessment of free text answers using a modified BLEU algorithm , 2011, Comput. Educ..

[9]  Mohd Juzaiddin Ab Aziz,et al.  Automatic Essay Grading System for Short Answers in English Language , 2013, J. Comput. Sci..

[10]  Salvatore Valenti,et al.  An Overview of Current Research on Automated Essay Grading , 2003, J. Inf. Technol. Educ..

[11]  Fátima Rodrigues,et al.  Automatic Assessment of Short Free Text Answers , 2012, CSEDU.

[12]  Ramlan Mahmod,et al.  Automated Marking System for Short Answer examination (AMS-SAE) , 2009, 2009 IEEE Symposium on Industrial Electronics & Applications.

[13]  Rosheena Siddiqi,et al.  Improving Teaching and Learning through Automated Short-Answer Marking , 2010, IEEE Transactions on Learning Technologies.

[14]  Wael Hassan Gomaa,et al.  Automatic scoring for answers to Arabic test questions , 2014, Comput. Speech Lang..

[15]  Rada Mihalcea,et al.  Text-to-Text Semantic Similarity for Automatic Short Answer Grading , 2009, EACL.

[16]  Stephen Pulman,et al.  Automarking: using computational linguistics to score short‚ free−text responses , 2003 .