ESTIMATING RELIABILITY FROM A SINGLE ADMINISTRATION OF A CRITERION-REFERENCED TEST*

A test in which the range of possible scores is partitioned into k non-overlapping intervals that define different levels of student mastery of a well-specified content domain is generally classified as criterion-referenced (Millman, 1974). An example is the familiar pass-fail test with a criterion of, say, 75 percent correct. Since such tests are often used in conjunction with instructional programs that attempt to maximize the number of students attaining the highest mastery states and thereby minimize the variability of test scores, the classical correlation between scores on parallel tests (or equivalently, the ratio of true to observed variance) may be attenuated by lack of variability and is unsatisfactory as an indicator of reliability (Popham & Husek, 1969). In addition, Hambleton and Novick (1973) note that the classical approach to reliability is based on a definition of error that is not entirely appropriate for such tests.

[1]  H. Huynh.  ON THE RELIABILITY OF DECISIONS IN DOMAIN‐REFERENCED TESTING , 1976 .

[2]  Edward H. Haertel,et al.  A Single-Administration Reliability Index for Criterion-Referenced Tests: The Mean Split-Half Coefficient of Agreement. , 1975 .

[3]  Ronald K. Hambleton,et al.  RELIABILITY OF CRITERION‐REFERENCED TESTS: A DECISION‐THEORETIC FORMULATION , 1974 .

[4]  Samuel A. Livingston.  A NOTE ON THE INTERPRETATION OF THE CRITERION‐REFERENCED RELIABILITY COEFFICIENT , 1973 .

[5]  Ronald K. Hambleton,et al.  TOWARD AN INTEGRATION OF THEORY AND METHOD FOR CRITERION-REFERENCED TESTS1,2 , 1973 .

[6]  C. Harris.  NOTE ON THE VARIANCES AND COVARIANCES OF THREE ERROR TYPES , 1973 .

[7]  M. R. Novick,et al.  The estimation of proportions inm groups , 1973 .

[8]  Samuel Livingston.  REPLY TO SHAVELSON, BLOCK, AND RAVITCH'S “CRITERION‐REFERENCED TESTING: COMMENTS ON RELIABILITY” , 1972 .

[9]  Richard J. Shavelson,et al.  CRITERION‐REFERENCED TESTING: COMMENTS ON RELIABILITY1 , 1972 .

[10]  C. Harris.  An Index of Efficiency for Fixed-Length Mastery Tests. , 1972 .

[11]  Samuel A. Livingston.  A REPLY TO HARRIS “AN INTERPRETATION OF LIVINGSTON'S RELIABILITY COEFFICIENT FOR CRITERION-REFERENCED TESTS” , 1972 .

[12]  Samuel Livingston,et al.  CRITERION‐REFERENCED APPLICATIONS OF CLASSICAL TEST THEORY 1,2 , 1972 .

[13]  C. Harris.  AN INTERPRETATION OF LIVINGSTON'S RELIABILITY COEFFICIENT FOR CRITERION‐REFERENCED TESTS , 1972 .

[14]  W. James Popham,et al.  Criterion-Referenced Measurement , 1971 .

[15]  T. R. Husek,et al.  IMPLICATIONS OF CRITERION‐REFERENCED MEASUREMENT1,2 , 1969 .

[16]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[17]  Frederic M. Lord,et al.  A strong true-score theory, with applications , 1965 .

[18]  S. Gupta.  Probability Integrals of Multivariate Normal and Multivariate $t^1$ , 1963 .

[19]  F. Lord.  An approach to mental test theory , 1959 .