Understanding Examinees’ Responses to Items: Implications for Measurement

Examinees' thinking processes have become an increasingly important concern in testing. The responses processes aspect is a major component of validity, and contemporary tests increasingly involve specifications about the cognitive complexity of examinees' response processes. Yet, empirical research findings on examinees' cognitive processes are not often available either to provide evidence for validity or to guide the design or selection of items. In this article, studies and developments from the author's research program are presented to illustrate how empirical studies on examinees' thinking processes can impact item and test design.

[1]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[2]  D. Wiley,et al.  Latent partition analysis , 1967, Psychometrika.

[3]  G. H. Fischer,et al.  The linear logistic test model as an instrument in educational research , 1973 .

[4]  Susan E. Whitely,et al.  Solving verbal analogies: Some cognitive components of intelligence test items. , 1976 .

[5]  Susan E. Whitely,et al.  Relationships in Analogy Items: a Semantic Component of a Psychometric Task , 1977 .

[6]  S. Whitely Information-Processing on Intelligence Test Items: Some Response Components , 1977 .

[7]  Susan E. Whitely,et al.  Multicomponent latent trait models for ability tests , 1980 .

[8]  Susan E. Whitely,et al.  Modeling Aptitude Test Validity from Cognitive Components. , 1980 .

[9]  S. Whitely Construct validity: Construct representation versus nomothetic span. , 1983 .

[10]  S. Embretson A general latent trait model for response processes , 1984 .

[11]  H. Swaminathan,et al.  Detecting Differential Item Functioning Using Logistic Regression Procedures , 1990 .

[12]  Michael T. Kane,et al.  An argument-based approach to validity. , 1992 .

[13]  Robert J. Mislevy,et al.  How to Equate Tests With Little or No Data , 1993 .

[14]  R. Mislevy Evidence and inference in educational assessment , 1994 .

[15]  S. Messick Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[16]  Susan E. Embretson,et al.  Generating items during testing: Psychometric issues and models , 1999 .

[17]  Matthias von Davier,et al.  A General Diagnostic Model Applied to Language Testing Data. Research Report. ETS RR-05-16. , 2005 .

[18]  Susan E. Embretson,et al.  Construct Validity: A Universal Validity System or Just Another Test Evaluation Procedure? , 2007 .

[19]  William Stout,et al.  Skills Diagnosis Using IRT‐Based Continuous Latent Trait Models , 2007 .

[20]  Robert W. Lissitz,et al.  A Suggested Change in Terminology and Emphasis Regarding Validity and Education , 2007 .

[21]  Susan E. Embretson,et al.  Understanding and Quantifying Cognitive Complexity Level in Mathematical Problem Solving Items , 2008 .

[22]  Susan E. Embretson,et al.  Designing Cognitive Complexity in Mathematical Problem-Solving Items , 2010 .

[23]  Xiangdong Yang,et al.  A Multicomponent Latent Trait Model for Diagnosis , 2013, Psychometrika.

[24]  The Multicomponent Latent Trait Model for Diagnosis , 2015, Applied psychological measurement.

[25]  Susan Embretson An Integrative Framework for Construct Validity , 2016 .