Appropriateness Measurement: Validating Studies and Variable Ability Models

Publisher Summary This chapter discusses the objectives and limitations of appropriateness measurement and three problems in appropriateness measurement. Appropriateness measurement is a general approach to the problem of inappropriate test scores. Its purpose is simply to identify inappropriate test scores and not to correct the causes of inappropriateness. It is limited to cases like those noted in which inappropriate test scores and unusual answer patterns tend to occur together. Appropriateness measurement is implemented by statistics, called appropriateness indices, that measure the degree to which an examinee's answer pattern is unusual, that is, unlike the pattern expected from typical examinees. The chapter discusses three problems left unanswered by the Levine and Rubin study, namely, (1) estimated versus known item parameters: how seriously will appropriateness measurement be affected by estimation errors?, (2) unidentified aberrant: how will the presence of unidentified aberrants affect parameter estimation and, consequently, appropriateness measurement?, and (3) model validity: simulated data conform precisely to the psychometric model used to generate data and to formulate appropriateness indices, but there will be reliable contradictions to the assumptions of any tractable, nontrivial psychometric model in a large sample of actual data.