The Application of the Cumulative Logistic Regression Model to Automated Essay Scoring

Most automated essay scoring programs use a linear regression model to predict an essay score from several essay features. This article applied a cumulative logit model instead of the linear regression model to automated essay scoring. Comparison of the performances of the linear regression model and the cumulative logit model was performed on a large variety of data sets. It appears that the cumulative logit model performed somewhat better than did the linear regression model.

[1]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[2]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[3]  Brent Bridgeman,et al.  Considering Fairness and Validity in Evaluating Automated Scoring , 2009 .

[4]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[5]  J. Fleiss,et al.  Quantification of agreement in psychiatric diagnosis revisited. , 1987, Archives of general psychiatry.

[6]  Gerald J. Hahn,et al.  Applied Regression Analysis (2nd Ed.) , 2012 .

[7]  Shelby J. Haberman,et al.  Conditional Log-Linear Models for Analyzing Categorical Panel Data , 1994 .

[8]  S. Haberman.  Concavity and estimation , 1989 .

[9]  J. Pratt.  Concavity of the Log Likelihood , 1981 .

[10]  Martin Chodorow,et al.  Automated Essay Evaluation: The Criterion Online Writing Service , 2004, AI Mag..

[11]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[12]  Valen E. Johnson,et al.  On Bayesian Analysis of Multirater Ordinal Data: An Application to Automated Essay Grading , 1996 .

[13]  J. Fleiss,et al.  Quantification of agreement in psychiatric diagnosis. A new approach. , 1967, Archives of general psychiatry.

[14]  Neil J. Dorans,et al.  IMPROVING THE STATISTICAL ASPECTS OF E‐RATER®: EXPLORING ALTERNATIVE FEATURE REDUCTION AND COMBINATION RULES , 2003 .

[15]  Shelby J. Haberman,et al.  When Can Subscores Have Value? , 2008 .

[16]  J. H. Schuenemeyer,et al.  Generalized Linear Models (2nd ed.) , 1992 .

[17]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .