Computerization of the examination process has facilitated fast and unbiased large-scale evaluation, by using the popular format of multiple choice based question papers. The primary drawback of this format is the possibility of guessing by the students. This has been largely remedied by online Computerized Adaptive Testing (CAT) methods where the next question presented to a student depends upon his/her responses to earlier questions of known difficulty levels. The student capabilities are evaluated in real time. However, the case where the questions’ difficulty levels are not apriori known, but in fact estimated aposteriori with respect to aggregate student performance, has been less analyzed. This paper addresses the problem of offline estimation of the student capability levels using a maximum likelihood estimation method and proposes to use these estimated capability levels rather than raw-marks for the purpose of ranking of students. We focus on the case when guessing is ruled out, for example, due to exact numerical answer possibilities instead of multiple choices. With respect to suitable performance objectives, like low errors in rank-allotment, the estimated-capability based ranking emerges more reliable than the traditional marks based ranking of students.
[1]
Prateek Jain,et al.
Low-rank matrix completion using alternating minimization
,
2012,
STOC '13.
[2]
F. Baker.
The basics of item response theory
,
1985
.
[3]
R. Karandikar.
On multiple choice tests and negative marking
,
2010
.
[4]
M. Valcke,et al.
Scoring methods for multiple choice assessment in higher education – Is it still a matter of number right scoring or negative marking?
,
2013
.
[5]
Shih-Chieh Huang,et al.
Web-based Assessment and Test Analyses (WATA) system: development and evaluation
,
2004,
J. Comput. Assist. Learn..
[6]
Derek C. Briggs,et al.
Generalizability in Item Response Modeling
,
2007
.