Ability Measure Equivalence of Computer Adaptive and Pencil and Paper Tests: A Research Synthesis.

ABSTRACT This paper reports on existing studies and uses meta analysis to compare and synthesize the results of 20 studies from 8 research reports comparing the ability measure equivalence of computer adaptive tests (CAT) and conventional paper and pencil tests. Using the research synthesis techniques developed by Hedges and Olkin (1985), it is possible to estimate a scale-free index of effect magnitude for each study and then compare results on a common scale. Most studies, despite differences in test content, age of examinees, item response theory model used, and study design, show comparable mean ability measures on CAT and pencil and paper test versions. Where mean differences are statistically significant, mean measures are higher for a pre-existing pencil and paper test. Three tables and two figures summarize the study results. (Contains 21 references.) (SLD)

[1]  James B. Olsen.  Applying Computerized Adaptive Testing in Schools. , 1990 .

[2]  Steven L. Wise,et al.  Research on the Effects of Administering Tests via Computers. , 1989 .

[3]  David J. Weiss,et al.  A Construct Validation of Adaptive Achievement Testing. , 1978 .

[4]  Bert F. Green,et al.  Critical Problems in Computer-Based Psychological Measurement , 1988 .

[5]  M E Lunz,et al.  Comparability of decisions for computer adaptive and written examinations. , 1991, Journal of allied health.

[6]  Anne L. Harvey,et al.  The Equivalence of Scores from Automated and Conventional Educational and Psychological Tests: A Review of the Literature. College Board Report No. 88-8. , 1988 .

[7]  Robert Cudeck,et al.  Adaptive and Conventional Versions of the DAT: The First Complete Test Battery Comparison , 1989 .

[8]  Bert F. Green.  Evaluation Plan for the Computerized Adaptive Vocational Aptitude Battery , 1982 .

[9]  R. Owen,et al.  A Bayesian Sequential Procedure for Quantal Response in the Context of Adaptive Mental Testing , 1975 .

[10]  Kathleen E. Moreno,et al.  Relationship Between Corresponding Armed Services Vocational Aptitude Battery (ASVAB) and Computerized Adaptive Testing (CAT) Subtests , 1984 .

[11]  Mark D. Reckase,et al.  Application of tailored testing to achievement measurement , 1977 .

[12]  Mark D. Reckase,et al.  TECHNICAL GUIDELINES FOR ASSESSING COMPUTERIZED ADAPTIVE TESTS , 1984 .

[13]  David C. Slawson,et al.  Comparison and equating of paper - administered, computer - administered and computerized adaptive t , 1986 .

[14]  J. B. Olsen,et al.  THE FOUR GENERATIONS OF COMPUTERIZED EDUCATIONAL MEASUREMENT , 1988 .

[15]  Mark D. Reckase,et al.  An interactive computer program for tailored testing based on the one-parameter logistic model , 1974 .