A Sampling Model for Validity

A multifacet sampling model, based on generalizability theory, is developed for the measurement of dispositional attributes. Dispositions are defined in terms of universes of observations, and the value of the disposition is given by the universe score, the mean over the universe defining the disposition. Observed scores provide estimates of universe scores, and errors of measurement are introduced in order to maintain consistency in these estimates. The sampling model provides a straightforward interpretation of validity in terms of the accuracy of estimates of the universe scores, and of reliability in terms of the consistency among these estimates. A third property of measurements, import, is defined in terms of all of the implications of a measurement. The model provides the basis for a detailed analysis of standardization and of the systematic errors that standardization creates; for example, the hypothesis that increases in reliability may cause decreases in validity is easily derived from the model. The model also suggests an explicit mechanism for relating the refinement of measurement procedures to the development of laws and theories.

[1]  Jaap Van Brakel,et al.  Foundations of measurement , 1983 .

[2]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[3]  J. W. Humberston.  Classical mechanics , 1980, Nature.

[4]  B. Wright,et al.  Best test design , 1979 .

[5]  Philip L. Smith.  Sampling Errors of Variance Components in Small Sample Multifacet Generalizability Studies , 1978 .

[6]  K. Jöreskog.  Structural analysis of covariance and correlation matrices , 1978 .

[7]  Roderick P. McDonald,et al.  Generalizability in Factorable Domains: "Domain Validity and Generalizability"1 , 1978 .

[8]  Bryan W. Coyle,et al.  A Review And Critique Of Analyses Of Multitrait-Multimethod Matrices. , 1977, Multivariate behavioral research.

[9]  R. Brennan,et al.  The Generalizability of Class Means , 1977 .

[10]  H. H. Clark.  Psychology and language , 1977 .

[11]  Linda Allal,et al.  THE SYMMETRY OF GENERALIZABILITY THEORY: APPLICATIONS TO EDUCATIONAL MEASUREMENT , 1976 .

[12]  I. Lakatos.  Falsification and the Methodology of Scientific Research Programmes , 1976 .

[13]  H. Kaiser,et al.  Domain Validity and Generalizability , 1975 .

[14]  Robert L. Ebel.  Explorations in Reliability Theory. , 1974 .

[15]  B. J. Winer,et al.  Statistical Principles in Experimental Design, 2nd Edition. , 1973 .

[16]  J. S. Wiggins,et al.  Personality and Prediction: Principles of Personality Assessment , 1973 .

[17]  Eva Nick,et al.  The dependability of behavioral measurements: theory of generalizability for scores and profiles , 1973 .

[18]  Robert H. Ennis,et al.  Philosophy of educational research , 1973 .

[19]  Donald B. Rubin,et al.  The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. , 1974 .

[20]  Glenn L. Rowley,et al.  Criterion-Referenced Measurement , 1972 .

[21]  Richard C. Anderson.  How to Construct Achievement Tests to Assess Comprehension , 1972 .

[22]  Imre Lakatos,et al.  Criticism and the Growth of Knowledge , 1972 .

[23]  Gerald M. Gillmore,et al.  A NOTE ON “ACCURACY” AND “PRECISION” , 1971 .

[24]  Donald W. Fiske,et al.  Measuring the concepts of personality , 1971 .

[25]  Ernest Nagel.  Observation And Theory In Science , 1971 .

[26]  MAURICE CROSLAND,et al.  Foundations of Science , 1970, Nature.

[27]  Leroy Wolins,et al.  A Procedure for Estimation of Trait, Method, and Error Variance Attributable to a Measure1 2 , 1970 .

[28]  John R. Bormuth,et al.  On the theory of achievement test items , 1970 .

[29]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[30]  Edward Mackinnon.  Aspects of Scientific Explanation: and Other Essays in the Philosophy of Science , 1967 .

[31]  Ma. de la Natividad Jiménez Salas,et al.  The Conduct of Inquiry , 1967 .

[32]  William W. Rozeboom,et al.  Foundations of the theory of prediction , 1966 .

[33]  J. Loevinger,et al.  PERSON AND POPULATION AS PSYCHOMETRIC CONCEPTS. , 1965, Psychological review.

[34]  Lee J. Cronbach,et al.  The Signal/Noise Ratio in the Comparison of Reliability Coefficients , 1964 .

[35]  Robert H. Ennis.  Operational Definitions* , 1964 .

[36]  K. Popper,et al.  Conjectures and refutations;: The growth of scientific knowledge , 1972 .

[37]  Eugene Galanter,et al.  Handbook of mathematical psychology: I. , 1963 .

[38]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[39]  K. Renner.  Must all tests be valid , 1962 .

[40]  Gilbert C. Finlay.  The Physical Science Study Committee , 1962, The School Review.

[41]  H. C. Corben,et al.  Classical Mechanics (2nd ed.) , 1961 .

[42]  Harold P. Bechtoldt,et al.  Construct validity: A Critique. , 1959 .

[43]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[44]  R. Tryon,et al.  Reliability and behavior domain validity: reformulation and historical critique. , 1957, Psychological bulletin.

[45]  J. Tukey,et al.  AVERAGE VALUES OF MEAN SQUARES IN FACTORIALS , 1956 .

[46]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[47]  S. Toulmin,et al.  Philosophy of Science , 1970, The British Journal for the History of Science.

[48]  H. Feigl,et al.  Readings in the Philosophy of Science , 1954 .

[49]  E. F. Lindquist,et al.  Design and analysis of experiments in psychology and education , 1953 .

[50]  Norman Campbell,et al.  What is Science , 1953 .

[51]  Carl G. Hempel,et al.  Fundamentals of Concept Formation in Empirical Science , 1952 .

[52]  P. Meehl.  On the circularity of the law of effect. , 1950 .

[53]  Henry S. Leonard,et al.  Testability and Meaning. , 1937 .

[54]  H. Jeffreys.  The Logic of Modern Physics , 1928, Nature.