Interval estimation under two study designs for kappa with binary classifications.

Cornfield's test-based method of setting a confidence interval on a parameter associated with a two-by-two contingency table is adapted for use with the measure of agreement kappa. One-sided confidence intervals derived in this way are compared to other intervals proposed for kappa under two study designs. Both designs involve two ratings per subject on a dichotomous scale. In one design the same two raters make all evaluations; in the other, possibly different pairs of raters evaluate different subjects, or the same rater carries out a pair of independent assessments for each subject. It is shown through simulation that lower bounds based on Cornfield's test-based method attain the nominal coverage probability more often than other intervals proposed in the literature.

[1]  Stephen E. Fienberg,et al.  Discrete Multivariate Analysis: Theory and Practice , 1976 .

[2]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[3]  H. Kraemer.  Ramifications of a population model forκ as a coefficient of reliability , 1979 .

[4]  J. Fleiss.  Statistical methods for rates and proportions , 1974 .

[5]  David R. Kincaid,et al.  Numerical mathematics and computing , 1980 .

[6]  H. Kraemer,et al.  2 x 2 kappa coefficients: measures of agreement or association. , 1989, Biometrics.

[7]  J. Fleiss,et al.  Inference About Weighted Kappa in the Non-Null Case , 1978 .

[8]  B. Everitt,et al.  Large sample standard errors of kappa and weighted kappa. , 1969 .

[9]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[10]  V. Flack.  Confidence intervals for the interrater agreement measure kappa , 1987 .

[11]  O. Miettinen,et al.  Estimability and estimation in case-referent studies. , 1976, American journal of epidemiology.

[12]  J. Fleiss,et al.  Jackknifing functions of multinomial frequencies, with an application to a measure of concordance. , 1982, American journal of epidemiology.

[13]  Jerome Cornfield,et al.  A Statistical Problem Arising from Retrospective Studies , 1956 .

[14]  J. B. Garner,et al.  The standard error of Cohen's Kappa. , 1991, Statistics in medicine.

[15]  J. Bigger.  Longitudinal (Natural History) Studies of Silent Myocardial Ischemia , 1988 .

[16]  H. L. Le Roy,et al.  Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability; Vol. IV , 1969 .