Validity and the Consequences of Test Interpretation and Use

The vast majority of measures have, at their core, a purpose of personal and social change. If test developers and users want measures to have personal and social consequences and impact, then it is critical to consider the consequences and side effects of measurement in the validation process itself. The consequential basis of test interpretation and use, as introduced in Messick’s (Educational measurement, Macmillan, New York, pp. 13–103, 1989) progressive matrix model of unified validity theory, has been misunderstood by many measurement experts, test developers, researchers, and practitioners. The purposes of this paper were to (a) review Messick’s unified view of validity and clarify his consequential basis of test interpretation and use, (b) discuss the kinds of questions evoked by value implications and social consequences and their role in construct validity and score meaning, (c) present a reframing of Messick’s model and a new model of unified validity and validation, (d) bring the concept of multilevel measures under the same validation umbrella as individual differences measures, and (e) offer some thoughts and directions for more explicit consideration of value implications, intended social consequences, and unintended side effects of legitimate test interpretation and use. This paper has implications for the interpretation, use, and validation of both individual differences and multilevel measures in education, psychology, and health contexts.

[1]  W. Popham.  Consequential validity: Right Concern‐Wrong Concept , 2005 .

[2]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[3]  S. Messick.  Test validity and the ethics of assessment. , 1980 .

[4]  D. Eignor.  The standards for educational and psychological testing. , 2013 .

[5]  Robert L. Linn,et al.  Evaluating the Validity of Assessments: The Consequences of Use , 2005 .

[6]  Lorrie A. Shepard,et al.  The Centrality of Test Use and Consequences for Test Validity. , 2005 .

[7]  R. Linn.  Educational measurement, 3rd ed. , 1989 .

[8]  Heather H. Koons,et al.  Sources of Validity Evidence for Educational and Psychological Tests , 2008 .

[9]  J. Loevinger.  Objective Tests as Instruments of Psychological Theory , 1957 .

[10]  S. Messick.  Test Validity: A Matter of Consequence , 1998 .

[11]  B. Zumbo,et al.  Validation of Multilevel Constructs: Validation Methods and Empirical Findings for the EDI , 2011 .

[12]  D. Borsboom.  Educational Measurement (4th ed.) , 2009 .

[13]  Bruno D. Zumbo,et al.  3 Validity: Foundational Issues and Statistical Methodology , 2006 .

[14]  Samuel Messick,et al.  CONSEQUENCES OF TEST INTERPRETATION AND USE: THE FUSION OF VALIDITY AND VALUES IN PSYCHOLOGICAL ASSESSMENT , 1998 .

[15]  A. Anastasi.  Evolving Concepts of Test Validation , 1986 .

[16]  William A. Mehrens,et al.  The Consequences of Consequential Validity. , 2005 .

[17]  H. V. Gelder.  The Netherlands , 2004, Constitutions of Europe (2 vols.).

[18]  L. Radloff.  The CES-D Scale , 1977 .

[19]  David E. Wiley,et al.  The role of constructs in psychological and educational measurement , 2001 .

[20]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[21]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[22]  R. Linn.  Validity of Inferences from Test-based Educational Accountability Systems , 2007 .

[23]  Robert W. Lissitz,et al.  The concept of validity : revisions, new directions, and applications , 2009 .

[24]  N. Cole,et al.  Gender and fair assessment , 1997 .

[25]  B. Zumbo,et al.  Testing and measurement from a multilevel view: Psychometrics and validation. , 2011 .

[26]  B. Zumbo,et al.  A Dialectic on Validity: Where We Have Been and Where We Are Going , 1996 .

[27]  Marise Ph. Born,et al.  The Role of Constructs in Psychological and Educational Measurement , 2002 .

[28]  Susy Macqueen,et al.  Validity , 1973, Just Algorithms.