The Generalizability of Class Means

Recently, a number of researchers have given serious consideration to the problem of estimating reliability when the unit of analysis is a class mean or some other aggregate score for a set of persons. Haney (1974a, 1974b) has reviewed important aspects of the relevant literature. The study of this topic has been motivated by the analysis of data from several different sources. In particular, large scale evaluations, such as those, undertaken for Head Start (see Smith & Bissell, 1970) and Follow Through (see Abt Associates, 1974), frequently require estimates of reliability when class is the unit of analysis. Similar issues arise in the study of course evaluation questionnaires (See Kane, Gillmore, & Crooks, 1976), and studies of school effectiveness and accountability (see Dyer, Linn, & Patton, 1969; Marco, 1974). Using concepts from classical reliability theory, Shaycoft (1962), Wiley (1970), and Thrash and Porter (Note 1) have de-

[1]  M. Kane,et al.  STUDENT EVALUATIONS OF TEACHING: THE GENERALIZABILITY OF CLASS MEANS , 1976 .

[2]  R. Brennan.  The Calculation of Reliability From a Split-Plot Factorial Design , 1975 .

[3]  Andrew C. Porter,et al.  Invalidity of a Current Method for Estimating Reliability. , 1974 .

[4]  B. J. Winer,et al.  Statistical Principles in Experimental Design, 2nd Edition. , 1973 .

[5]  Eva Nick,et al.  The dependability of behavioral measurements: theory of generalizability for scores and profiles , 1973 .

[6]  Donald B. Rubin,et al.  The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. , 1974 .

[7]  Marshall S. Smith,et al.  Report Analysis: The Impact of Head Start. , 1970 .

[8]  David E. Wiley,et al.  The evaluation of instruction : issues and problems , 1970 .

[9]  R. Kirk.  Experimental Design: Procedures for the Behavioral Sciences , 1970 .

[10]  David E. Wiley.  The Design and Analysis of Evaluation Studies. , 1969 .

[11]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[12]  G. Glass,et al.  RULES OF THUMB FOR WRITING THE ANOVA TABLE , 1967 .

[13]  L. Cronbach,et al.  Intraclass Correlation as an Approximation to the Coefficient of Generalizability , 1964 .

[14]  J. Tukey,et al.  AVERAGE VALUES OF MEAN SQUARES IN FACTORIALS , 1956 .

[15]  E. F. Lindquist,et al.  Design and analysis of experiments in psychology and education , 1953 .

[16]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[17]  C. Hoyt.  Test reliability estimated by analysis of variance , 1941 .

[18]  M. W. Richardson,et al.  The theory of the estimation of test reliability , 1937 .