Test Design and Speededness

A critical component of test speededness is the distribution of the test taker’s total time on the test. A simple set of constraints on the item parameters in the lognormal model for response times is derived that can be used to control the distribution when assembling a new test form. As the constraints are linear in the item parameters, they can easily be included in a mixed integer programming model for test assembly. The use of the constraints is demonstrated for the problems of assembling a new test form to be equally speeded as a reference form, test assembly in which the impact of a change in the content specifications on speededness is to be neutralized, and the assembly of test forms with a revised level of speededness.

[1]  R. H. Klein Entink,et al.  A Multivariate Multilevel Approach to the Modeling of Accuracy and Speed of Test Takers , 2008, Psychometrika.

[2]  Wim J. van der Linden,et al.  Bayesian Procedures for Identifying Aberrant Response-Time Patterns in Adaptive Testing , 2008 .

[3]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[4]  Willem J. van der Linden,et al.  A lognormal model for response times on test items , 2006 .

[5]  H. Gulliksen.  Theory of mental tests , 1952 .

[6]  Wim J. van der Linden.  Linking Response-Time Parameters onto a Common Scale. , 2010 .

[7]  L. Cronbach,et al.  Time-limit tests: Estimating their reliability and degree of speeding , 1951, Psychometrika.

[8]  J WIM,et al.  A HIERARCHICAL FRAMEWORK FOR MODELING SPEED AND ACCURACY ON TEST ITEMS , 2007 .

[9]  W. D. Linden,et al.  Conceptual Issues in Response-Time Modeling. , 2009 .

[10]  Brent Bridgeman,et al.  Effects of Differentially Time-Consuming Tests on Computer-Adaptive Test Scores , 2004 .

[11]  ten Josephus Berge,et al.  Encyclopedia of Statistical Sciences, Update Volume 3 , 2000 .

[12]  G. Zuriff,et al.  Extra Examination Time for Students With Learning Disabilities: An Examination of the Maximum Potential Thesis , 2000 .

[13]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[14]  Susan Ellerin Rindler,et al.  PITFALLS IN ASSESSING TEST SPEEDEDNESS , 1979 .

[15]  Krista Breithaupt,et al.  Detecting Differential Speededness in Multistage Testing , 2007 .

[16]  Wim J. van der Linden,et al.  Using Response-Time Constraints to Control for Differential Speededness in Computerized Adaptive Testing , 1999 .

[17]  W. D. Linden.  Predictive Control of Speededness in Adaptive Testing , 2009 .

[18]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[19]  Jean-Paul Fox,et al.  Modeling of Responses and Response Times with the Package cirt , 2007 .

[20]  E. Lehmann.  Elements of large-sample theory , 1998 .

[21]  J. Naus.  The Distribution of the Logarithm of the Sum of Two Log-Normal Variates , 1969 .

[22]  Won-Chan Lee,et al.  Classification Consistency and Accuracy for Complex Assessments Using Item Response Theory , 2010 .

[23]  W. D. Linden.  Setting Time Limits on Tests. , 2011 .

[24]  Jean-Paul Fox,et al.  Evaluating cognitive theory: a joint modeling approach using responses and response times. , 2009, Psychological methods.

[25]  Jason W. Osborne,et al.  Extended Time Test Accommodation: Directions for Future Research and Practice , 2005 .

[26]  Stephen G. Sireci,et al.  Test Accommodations for Students With Disabilities: An Analysis of the Interaction Hypothesis , 2005 .

[27]  Willem J. van der Linden,et al.  Linear Models for Optimal Test Design , 2005 .

[28]  L. Fenton.  The Sum of Log-Normal Probability Distributions in Scatter Transmission Systems , 1960 .

[29]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .