Validity Issues in Computer-Based Testing

Advances in technology are stimulating the development of complex, computerized assessments. The prevailing rationales for developing computer-based assessments are improved measurement and increased efficiency. In the midst of this measurement revolution, test developers and evaluators must revisit the notion of validity. In this article, we discuss the potential positive and negative effects computer-based testing could have on validity, review the literature regarding validation perspectives in computer-based testing, and provide suggestions regarding how to evaluate the contributions of computer-based testing to more valid measurement practices. We conclude that computer-based testing shows great promise for enhancing validity, but at this juncture, it remains equivocal whether technological innovations in assessment have led to more valid measurement.

[1]  Robert L. Linn,et al.  Evaluating the Validity of Assessments: The Consequences of Use , 2005 .

[2]  R. Linn.  Partitioning Responsibility for the Evaluation of the Conseqyences of Assessment Programs , 2005 .

[3]  Stephen G. Sireci,et al.  Technological Innovations in Large-Scale Assessment , 2002 .

[4]  Susan E. Embretson,et al.  Generating items during testing: Psychometric issues and models , 1999 .

[5]  Wim J. van der Linden,et al.  Using Response-Time Constraints to Control for Differential Speededness in Computerized Adaptive Testing , 1999 .

[6]  Randy Elliot Bennett,et al.  Psychometric and Cognitive Functioning of an Under-Determined Computer-Based Response Type for Quantitative Reasoning , 1999 .

[7]  David M. Williamson,et al.  "Mental Model" Comparison of Automated and Human Scoring , 1999 .

[8]  Irwin S. Kirsch,et al.  Examining the Relationship Between Computer Familiarity and Performance on Computer‐Based Language Tasks , 1999 .

[9]  Randy Elliot Bennett,et al.  USING NEW TECHNOLOGY TO IMPROVE ASSESSMENT , 1999 .

[10]  S. Sireci.  The Construct of Content Validity , 1998 .

[11]  Willem J. van der Linden,et al.  Optimal Assembly of Psychological and Educational Tests , 1998 .

[12]  B. Veldkamp.  Multiple Objective Test Assembly Problems , 1998 .

[13]  John F. Booth,et al.  The User Interface in Computer-based Selection and Assessment: Applied and Theoretical Problematics of an Evolving Technology , 1998 .

[14]  J. Booth.  Uses of PC Technology in Selection and Assessment , 1998 .

[15]  Thomas E. Piemme,et al.  Development of a Scoring Algorithm to Replace Expert Rating for Scoring a Complex Performance-Based Assessment , 1997 .

[16]  Tianyou Wang,et al.  Computerized Adaptive and Fixed‐Item Testing of Music Listening Skill: A Comparison of Efficiency, Precision, and Concurrent Validity , 1997 .

[17]  Marc M. Sebrechts,et al.  A Computer-Based Task for Measuring the Representational Component of Quantitative Proficiency. , 1997 .

[18]  Stephen G. Clyman,et al.  Scoring a Performance-Based Assessment by Modeling the Judgments of Experts , 1995 .

[19]  Randy Elliot Bennett,et al.  Generalizability, Validity, and Examinee Perceptions of a Computer‐Delivered Formulating‐Hypotheses Test , 1995 .

[20]  S. Messick.  The Interplay of Evidence and Consequences in the Validation of Performance Assessments , 1994 .

[21]  M. Stocking,et al.  A Model and Heuristic For Solving Very Large Item Selection Problems , 1993 .

[22]  Lorrie A. Shepard,et al.  Chapter 9: Evaluating Test Validity , 1993 .

[23]  Randy Elliot Bennett,et al.  A REVIEW OF AUTOMATICALLY SCORABLE CONSTRUCTED-RESPONSE ITEM TYPES FOR LARGE-SCALE ASSESSMENT , 1992 .

[24]  Michael T. Kane,et al.  An argument-based approach to validity. , 1992 .

[25]  Martha L. Stocking,et al.  A Method for Severely Constrained Item Selection in Adaptive Testing , 1992 .

[26]  Howard Wainer,et al.  SOME PRACTICAL CONSIDERATIONS WHEN CONVERTING A LINEARLY ADMINISTERED TEST TO AN ADAPTIVE FORMAT , 1992 .

[27]  Isaac I. Bejar,et al.  A methodology for scoring open-ended architectural design problems. , 1991 .

[28]  H. Wainer,et al.  Toward a Psychometrics for Testlets , 1989 .

[29]  R. Darrell Bock,et al.  Item Pool Maintenance in the Presence of Item Parameter Drift. , 1988 .

[30]  Howard Wainer,et al.  Item Clusters and Computerized Adaptive Testing: A Case for Testlets , 1987 .

[31]  Bert F. Green,et al.  Equivalence of Conventional and Computer Presentation of Speed Tests , 1986 .

[32]  Gerhard H. Fischer,et al.  Some Applications of Logistic Latent Trait Models with Linear Constraints on the Parameters , 1982 .

[33]  Fritz Drasgow,et al.  Innovations in Computerized Assessment , 1999 .

[34]  L. Shepard.  Chapter 9: Evaluating Test Validity , 1993 .

[35]  J. Booth,et al.  The key to valid computer-based testing : the user interface , 1991 .