1-27-2012 Statistical Issues in Assessing Hospital Performance

Preface The Centers for Medicare and Medicaid Services (CMS), through a subcontract with Yale New is supporting a committee appointed by the Committee of Presidents of Statistical Societies (COPSS) to address statistical issues identified by the CMS and stakeholders about CMS's approach to mod­ eling hospital quality based on outcomes. In the spring of 2011, with the direct support of YN­ HHSC/CORE, COPSS formed a committee comprised of one member from each of its constituent societies, a chair, and a staff member from the American Statistical Association, and held a prelim­ inary meeting in April. In June, YNHHSC/CORE executed a subcontract with COPSS under its CMS contract to support the development of a White Paper on statistical modeling. Specifically, YNHHSC/CORE contracted with COPSS to " provide guidance on statistical approaches. .. when estimating performance metrics, " and " consider and discuss concerns commonly raised by stake­ holders (hospitals, consumer, and insurers) about the use of " hierarchical generalized linear models in profiling hospital quality. The committee convened in June and August of 2011, and exchanged a wide variety of materials. To ensure the committee's independence, YNHHSC/CORE did not comment on the white paper findings, and CMS pre-cleared COPSS' publication of an academic manuscript based on the White Paper. The committee thanks COPSS and especially its chair, Xihong Lin of the Harvard School of Public Health; and staff of the American Statistical Association, especially Steve Pierson and Keith Crank, for their efforts in establishing the committee and coordinating its work. We thank Darcey Cobbs-Yale New Haven Hospital who issued the contract on behalf of CMS. COPSS developed a special formal review process for this report with the goals of ensuring that it is objective and addresses the CMS charge. Consequently, this report was reviewed in draft form by professionals with a broad range of perspectives and expertise. Xihong Lin coordinated the review. We thank her and the following individuals for donating their time and expertise: Adal­ charter member societies. The preamble to the COPSS charter states, " Whereas the various societies have distinct characteristics they also have some common interests and concerns that can benefit from coordinated effort. The purpose of the Committee of Presidents of Statistical Societies (COPSS) is to work on shared problems, to improve intersociety communication, and to offer distinguished awards. Other activities designed to promote common interests among the member societies may be undertaken from time to …

[1]  Jeffrey S. Simonoff,et al.  The SAGE Handbook of Multilevel Modeling , 2013 .

[2]  L. Goldman,et al.  The accuracy of present-on-admission reporting in administrative data. , 2011, Health services research.

[3]  Logistic versus hierarchical modeling: an analysis of a statewide inpatient sample. , 2011, Journal of the American College of Surgeons.

[4]  David J. Spiegelhalter,et al.  The Identification of “Unusual” Health-Care Providers From a Hierarchical Model , 2011 .

[5]  Thomas A Louis,et al.  Percentile‐based empirical distribution function estimates for performance evaluation of healthcare providers , 2011, Journal of the Royal Statistical Society. Series C, Applied statistics.

[6]  Yuedong Wang,et al.  Smoothing Splines: Methods and Applications , 2011 .

[7]  K. Coombes,et al.  What information should be required to support clinical "omics" publications? , 2011, Clinical chemistry.

[8]  S. Fienberg.  Bayesian Models and Methods in Public Policy and Government Settings , 2011, 1108.2177.

[9]  Harlan M. Krumholz,et al.  An Administrative Claims Measure Suitable for Profiling Hospital Performance Based on 30-Day All-Cause Readmission Rates Among Patients With Acute Myocardial Infarction , 2011, Circulation. Cardiovascular quality and outcomes.

[10]  P. Rosenbaum,et al.  The Hospital Compare mortality model and the volume-outcome relationship. , 2010, Health services research.

[11]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[12]  Gabriel J. Escobar,et al.  Effect of Choice of Estimation Method on Inter-Hospital Mortality Rate Comparisons , 2010, Medical care.

[13]  Z. Q. John Lu.  Bayesian methods for data analysis, third edition , 2010 .

[14]  S. Normand,et al.  Hospital volume and 30-day mortality for three common medical conditions. , 2010, The New England journal of medicine.

[15]  Hao Helen Zhang,et al.  Variable Selection for Semiparametric Mixed Models in Longitudinal Studies , 2010, Biometrics.

[16]  S. Fienberg,et al.  The Pleasures of Statistics: The Autobiography of Frederick Mosteller , 2010 .

[17]  J. Birkmeyer,et al.  Composite measures for predicting surgical mortality in the hospital. , 2009, Health affairs.

[18]  Thomas A Louis,et al.  Effective communication of standard errors and confidence intervals. , 2008, Biostatistics.

[19]  Thomas A. Louis,et al.  Ranking USRDS provider specific SMRs from 1998–2001 , 2008, Health Services and Outcomes Research Methodology.

[20]  D. Greiner.  Causal Inference in Civil Rights Litigation , 2008 .

[21]  Sharon-Lise T Normand,et al.  Comparison of “Risk-Adjusted” Hospital Outcomes , 2008, Circulation.

[22]  Yingye Zheng,et al.  Integrating the predictiveness of a marker with its performance as a classifier. , 2007, American journal of epidemiology.

[23]  Xihong Lin.  Estimation using penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models , 2007, Lifetime data analysis.

[24]  P J Diggle,et al.  Spatial modelling and the prediction of Loa loa risk: decision making under uncertainty , 2007, Annals of tropical medicine and parasitology.

[25]  Alan Y. Chiang,et al.  Generalized Additive Models: An Introduction With R , 2007, Technometrics.

[26]  Sharon-Lise T. Normand,et al.  Statistical and Clinical Aspects of Hospital Outcomes Profiling , 2007, 0710.4622.

[27]  Andrew Gelman.  Bayesian Checking of the Second Levels of Hierarchical Models. Comment.. , 2007 .

[28]  Thomas A. Louis,et al.  Flexible distributions for triple-goal estimates in two-stage hierarchical models , 2006, Comput. Stat. Data Anal..

[29]  Jeffrey S. Morris,et al.  Wavelet‐based functional mixed models , 2006, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[30]  Thomas A Louis,et al.  Loss Function Based Ranking in Two-Stage, Hierarchical Models. , 2006, Bayesian analysis.

[31]  P. Qiu.  The Statistical Evaluation of Medical Tests for Classification and Prediction , 2005 .

[32]  Geert Verbeke,et al.  Multiple Imputation for Model Checking: Completed‐Data Plots with Missing and Latent Data , 2005, Biometrics.

[33]  J. W. Thomas,et al.  Risk Adjustment for Measuring Health Care Outcomes, 3rd edition , 2004 .

[34]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[35]  Sharon-Lise T. Normand,et al.  Analytic Methods for Constructing Cross-Sectional Profiles of Health Care Providers , 2000, Health Services and Outcomes Research Methodology.

[36]  D. McCaffrey,et al.  Propensity score estimation with boosted regression for evaluating causal effects in observational studies. , 2004, Psychological methods.

[37]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[38]  Yeow Meng Thum,et al.  Setting Performance Standards: Concepts, Methods, and Perspectives , 2003 .

[39]  S. Normand,et al.  The volume-outcome relationship: from Luft to Leapfrog. , 2003, The Annals of thoracic surgery.

[40]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[41]  Thomas A Louis,et al.  Uncertainty in Rank Estimation: Implications for Value-Added Modeling Accountability Systems , 2002, Journal of educational and behavioral statistics : a quarterly publication sponsored by the American Educational Research Association and the American Statistical Association.

[42]  J. Birkmeyer,et al.  Hospital volume and surgical mortality in the United States. , 2002, The New England journal of medicine.

[43]  G. Kalton,et al.  Small-area income and poverty estimates : priorities for 2000 and beyond , 2000 .

[44]  A Milstein,et al.  Selective referral to high-volume hospitals: estimating potentially avoidable deaths. , 2000, JAMA.

[45]  M Buyse.  R(2): a useful measure of model performance when predicting a dichotomous outcome. , 2000, Statistics in medicine.

[46]  A. Ash,et al.  R2: a useful measure of model performance when predicting a dichotomous outcome. , 1999, Statistics in medicine.

[47]  T. Louis,et al.  Triple‐goal estimates in two‐stage hierarchical models , 1998 .

[48]  L. Iezzoni.  Assessing Quality Using Administrative Data , 1997, Annals of Internal Medicine.

[49]  S. Zeger,et al.  A Smooth Nonparametric Estimate of a Mixing Distribution Using Mixtures of Gaussians , 1996 .

[50]  J. Allison.  Risk adjustment for measuring health care outcomes , 1996 .

[51]  Tricia Walker,et al.  Computer science , 1996, English for academic purposes series.

[52]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[53]  Thomas J. Tomberlin,et al.  PREDICTING ACCIDENT FREQUENCIES FOR DRIVERS CLASSIFIED BY TWO FACTORS , 1988 .

[54]  M. Blumberg.  Comments on HCFA hospital death rate statistical outliers. Health Care Financing Administration. , 1987, Health services research.

[55]  D. Freedman,et al.  Regression Models for Adjusting the 1980 Census , 1986 .

[56]  Joseph B. Kadane,et al.  Estimating the Population in a Census Year 1980 and beyond , 1985 .

[57]  B. Efron.  Regression and ANOVA with Zero-One Data: Measures of Residual Variation , 1978 .

[58]  Stephen E. Fienberg,et al.  Discrete Multivariate Analysis: Theory and Practice , 1976 .

[59]  J. Bunker.  The national halothane study : a study of the possible association between halothane anesthesia and postoperative hepatic necrosis; report , 1971 .

[60]  L. Berkman,et al.  AMERICAN JOURNAL OF EPIDEMIOLOGY , 1965 .