Current Concerns in Validity Theory

We are at the end of the first century of work on models of educational and psychological measurement and into a new millennium. This certainly seems like an appropriate time for looking backward and looking forward in assessment. Furthermore, a new edition of the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999) has been published, and the previous editions of the Standards have served as benchmarks in the development of measurement theory. This backward glance will be just that, a glance. After a brief historical review focusing mainly on construct validity, the current state of validity theory will be summarized, with an emphasis on the role of arguments in validation. Then how an argument-based approach might be applied will be examined in regards to two issues in validity theory: the distinction between performance-based and theory-based interpretations, and the role of consequences in validation.

[1]  R. Linn.  Assessments and Accountability , 2000 .

[2]  James W. Pellegrino,et al.  Addressing the "Two Disciplines" Problem: Linking Theories of Cognition and Learning With Assessment and Instructional Practice , 1957 .

[3]  Jay P. Heubert,et al.  High stakes : testing for tracking, promotion, and graduation , 1999 .

[4]  S. Messick.  Test Validity: A Matter of Consequence , 1998 .

[5]  Suzanne Lane,et al.  A Framework for Evaluating the Consequences of Assessment Programs. , 1998 .

[6]  Allan S. Cohen,et al.  Threats to the Valid Use of Assessments , 1996 .

[7]  Michael T. Kane,et al.  The Precision of Measurements , 1996 .

[8]  D. Trauner,et al.  Behavioral Differences in School Age Children after Perinatal Stroke , 1996 .

[9]  M. Kane,et al.  Validating Interpretive Arguments for Licensure and Certification Examinations , 1994, Evaluation & the health professions.

[10]  L. Shepard.  Chapter 9: Evaluating Test Validity , 1993 .

[11]  Michael T. Kane,et al.  An argument-based approach to validity. , 1992 .

[12]  P. Moss.  Shifting Conceptions of Validity in Educational Measurement: Implications for Performance Assessment , 1992 .

[13]  Stephen B. Dunbar,et al.  Complex, Performance-Based Assessment: Expectations and Validation Criteria , 1991 .

[14]  M. Kane.  The Argument-Based Approach to Validation , 1990 .

[15]  Lee J. Cronbach,et al.  Construct validation after thirty years. , 1989 .

[16]  R. Snow,et al.  Implications of cognitive psychology for educational measurement. , 1989 .

[17]  L. Cronbach.  Five perspectives on the validity argument. , 1988 .

[18]  Hans Rott,et al.  'Falsification and the Methodology of Scientific Research Programmes' (Lakatos) , 1988 .

[19]  James W. Pellegrino,et al.  Mental models and mental tests. , 1988 .

[20]  W. W. Willingham.  TESTING HANDICAPPED PEOPLE—THE VALIDITY ISSUE , 1986 .

[21]  A. Anastasi.  Evolving Concepts of Test Validation , 1986 .

[22]  S. Whitely.  Construct validity: Construct representation versus nomothetic span. , 1983 .

[23]  M. Kane.  A Sampling Model for Validity , 1982 .

[24]  S. Messick.  Evidence and Ethics in the Evaluation of Tests , 1981 .

[25]  E. House.  Evaluating with Validity , 1982 .

[26]  S. Messick.  Test validity and the ethics of assessment. , 1980 .

[27]  R. Guion,et al.  On Trinitarian doctrines of validity. , 1980 .

[28]  L. Cronbach.  Selection Theory for a Political World , 1980 .

[29]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[30]  Mary L. Tenopyr,et al.  CONTENT?CONSTRUCT CONFUSION , 1977 .

[31]  R. Guion.  Content Validity—The Source of My Discontent , 1977 .

[32]  I. Lakatos.  Falsification and the Methodology of Scientific Research Programmes , 1976 .

[33]  S. Messick.  THE STANDARD PROBLEM: MEANING AND VALUES IN MEASUREMENT AND EVALUATION , 1974 .

[34]  F. Suppe.  The Structure of Scientific Theories. , 1975 .

[35]  Samuel Messick,et al.  THE STANDARD PROBLEM: MEANING AND VALUES IN MEASUREMENT AND EVALUATION , 1974 .

[36]  Eva Nick,et al.  The dependability of behavioral measurements: theory of generalizability for scores and profiles , 1973 .

[37]  Donald B. Rubin,et al.  The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. , 1974 .

[38]  I. Lakatos,et al.  Criticism and the Growth of Knowledge: Falsification and the Methodology of Scientific Research Programmes , 1970 .

[39]  M. Resnik,et al.  Aspects of Scientific Explanation. , 1966 .

[40]  C. Hempel,et al.  Aspects of Scientific Explanation and Other Essays in the Philosophy of Science. , 1966 .

[41]  K. Popper,et al.  Conjectures and refutations;: The growth of scientific knowledge , 1972 .

[42]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[43]  K. Renner.  Must all tests be valid , 1962 .

[44]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[45]  A. Flew,et al.  Patterns of Discovery. , 1961 .

[46]  L. Cronbach,et al.  Psychological tests and personnel decisions , 1958 .

[47]  J. Loevinger.  Objective Tests as Instruments of Psychological Theory , 1957 .

[48]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.