Kappa statistic for clustered dichotomous responses from physicians and patients

The bootstrap method for estimating the standard error of the kappa statistic in the presence of clustered data is evaluated. Such data arise, for example, in assessing agreement between physicians and their patients regarding their understanding of the physician-patient interaction and discussions. We propose a computationally efficient procedure for generating correlated dichotomous responses for physicians and assigned patients for simulation studies. The simulation result demonstrates that the proposed bootstrap method produces better estimate of the standard error and better coverage performance compared with the asymptotic standard error estimate that ignores dependence among patients within physicians with at least a moderately large number of clusters. We present an example of an application to a coronary heart disease prevention study.

[1]  D M Clarke,et al.  Comparing correlated kappas by resampling: is one level of agreement significantly different from another? , 1996, Journal of psychiatric research.

[2]  A Donner,et al.  Adjustments to the Mantel-Haenszel chi-square statistic and odds ratio variance estimator when the data are clustered. , 1987, Statistics in medicine.

[3]  Michael Cunningham,et al.  More than Just the Kappa Coefficient: A Program to Fully Characterize Inter-Rater Reliability between Two Raters , 2009 .

[4]  C. Field,et al.  Bootstrapping clustered data , 2007 .

[5]  J. Richard Landis,et al.  Large sample variance of kappa in the case of different sets of raters. , 1979 .

[6]  H. Kraemer,et al.  Extension of the kappa coefficient. , 1980, Biometrics.

[7]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[8]  Allan Donner,et al.  A Simulation Study of the Analysis of Sets of 2 × 2 Contingency Tables under Cluster Sampling: Estimation of a Common Odds Ratio , 1990 .

[9]  N L Oden.  Estimating kappa from binocular data. , 1991, Statistics in medicine.

[10]  Huiman X Barnhart,et al.  Weighted Least‐Squares Approach for Comparing Correlated Kappa , 2002, Biometrics.

[11]  A Donner,et al.  Testing the equality of two dependent kappa statistics. , 2000, Statistics in medicine.

[12]  Geert Molenberghs,et al.  Regression modelling of weighted κ by using generalized estimating equations , 2000 .

[13]  A Donner,et al.  Application of the McNemar test to non-independent matched pair data. , 1991, Statistics in medicine.

[14]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[15]  A K Manatunga,et al.  Modeling kappa for measuring dependent categorical agreement data. , 2000, Biostatistics.

[16]  S. Gross,et al.  The kappa coefficient of agreement for multiple observers when the number of subjects is small. , 1986, Biometrics.

[17]  W. Barlow,et al.  A comparison of methods for calculating a stratified kappa. , 1990, Statistics in medicine.

[18]  H J Schouten.  Estimating kappa from binocular data and comparing marginal probabilities. , 1993, Statistics in medicine.

[19]  Bahjat F. Qaqish,et al.  A family of multivariate binary distributions for simulating correlated binary variables with specified marginal means and correlations , 2003 .

[20]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[21]  Patrick Royston,et al.  The design of simulation studies in medical statistics , 2006, Statistics in medicine.

[22]  Neil Klar,et al.  An Estimating Equations Approach for Modelling Kappa , 2000 .

[23]  Stacey L. Sheridan,et al.  Patient–physician agreement on the content of CHD prevention discussions , 2011, Health expectations : an international journal of public participation in health care and health policy.

[24]  A. Albert,et al.  A bootstrap method for comparing correlated kappa coefficients , 2008 .

[25]  Jianwen Cai,et al.  Bootstrap analysis of multivariate failure time data , 2005, Statistics in medicine.

[26]  B. Efron,et al.  Bootstrap confidence intervals , 1996 .

[27]  T. Jackson,et al.  Incremental costs of hospital-acquired complications in Alberta, Canada , 2011, BMC Health Services Research.

[28]  Robert Tibshirani,et al.  Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy , 1986 .

[29]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .