Can valid and practical risk-prediction or casemix adjustment models, including adjustment for comorbidity, be generated from English hospital administrative data (Hospital Episode Statistics)? A national observational study

Abstract Background NHS hospitals collect a wealth of administrative data covering accident and emergency (AE hospital performance on national audit process measures largely correlated only with HF readmission rates. Conclusions Many practical risk-prediction or casemix adjustment models can be generated from HES data using LR, though an extra step is often required for accurate calibration. Including outpatient data in readmission models is useful. The three machine learning methods we assessed added little with these data. Readmission rates for HF patients should be divided by diagnosis on readmission when used for quality improvement. Future work As HES data continue to develop and improve in scope and accuracy, they can be used more, for instance A&E records. The return to theatre metric appears promising and could be extended to other index procedures and specialties. While our data did not warrant the testing of a larger number of machine learning methods, databases augmented with physiological and pathology information, for example, might benefit from methods such as boosted trees. Finally, one could apply the HF readmissions analysis to other chronic conditions. Funding

[1]  B. Manktelow,et al.  Comparison of four methods for deriving hospital standardised mortality ratios from a single hierarchical logistic regression model , 2016, Statistical methods in medical research.

[2]  Giovanni Montana,et al.  Classifier calibration using splined empirical probabilities in clinical risk prediction , 2015, Health care management science.

[3]  A. Bottle,et al.  Return to theatre for elective hip and knee replacements: what is the relative importance of patient factors, surgeon and hospital? , 2014, The bone & joint journal.

[4]  A. Bottle,et al.  Effect of the readmission primary diagnosis and time interval in heart failure patients: analysis of English administrative data , 2014, European journal of heart failure.

[5]  Harlan M Krumholz,et al.  Hospital readmission performance and patterns of readmission: retrospective cohort study of Medicare admissions , 2013, BMJ.

[6]  A. Bottle,et al.  Provider profiling models for acute coronary syndrome mortality using administrative data. , 2013, International journal of cardiology.

[7]  Ruben Amarasingham,et al.  Allocating scarce resources in real-time to reduce heart failure readmissions: a prospective, controlled study , 2013, BMJ quality & safety.

[8]  A. Majeed,et al.  Quality of routine hospital birth records and the feasibility of their use for creating birth cohorts. , 2013, Journal of public health.

[9]  A. Bottle,et al.  Systematic Review of Comorbidity Indices for Administrative Data , 2012, Medical care.

[10]  Carl van Walraven,et al.  Predicting the risk of unplanned readmission or death within 30 days of discharge after a heart failure hospitalization. , 2012, American heart journal.

[11]  Peter C Austin,et al.  Lifetime Analysis of Hospitalizations and Survival of Patients Newly Admitted With Heart Failure , 2012, Circulation. Heart failure.

[12]  P. Ziprin,et al.  Systematic review of discharge coding accuracy. , 2012, Journal of public health.

[13]  J. Calvin,et al.  Depression predicts repeated heart failure hospitalizations. , 2012, Journal of cardiac failure.

[14]  A. Bottle,et al.  Comorbidity scores for administrative data benefited from adaptation to local coding and diagnostic practices. , 2011, Journal of clinical epidemiology.

[15]  K. Carlsen,et al.  Non‐attendance rate in a Danish University Clinic of Dermatology , 2011, Journal of the European Academy of Dermatology and Venereology : JEADV.

[16]  A. Darzi,et al.  The volume–outcome relationship for radical cystectomy in England: an analysis of outcomes other than mortality , 2011, BJU international.

[17]  S. Pedersen,et al.  Symptoms of Anxiety and Cardiac Hospitalizations at 12 Months in Patients with Heart Failure , 2011, Journal of General Internal Medicine.

[18]  Mei-Chen Hu,et al.  Zero-Inflated and Hurdle Models of Count Data with Extra Zeros: Examples from an HIV-Risk Reduction Intervention Trial , 2011, The American journal of drug and alcohol abuse.

[19]  Ara Darzi,et al.  Variation in reoperation after colorectal surgery in England as an indicator of surgical performance: retrospective analysis of Hospital Episode Statistics , 2011, BMJ : British Medical Journal.

[20]  Sebastian Schneeweiss,et al.  A combined comorbidity score predicted mortality in elderly patients better than existing scores. , 2011, Journal of clinical epidemiology.

[21]  P. Gibbs,et al.  Predictors of clinic non‐attendance: opportunities to improve patient outcomes in colorectal cancer , 2010, Internal medicine journal.

[22]  M. Pfisterer,et al.  Predictors of early readmission or death in elderly patients with heart failure. , 2010, American heart journal.

[23]  K. Laupland,et al.  Validity of ascertainment of co-morbid illness using administrative databases: a systematic review. , 2010, Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases.

[24]  C. Ko,et al.  Risk adjustment in the American College of Surgeons National Surgical Quality Improvement Program: a comparison of logistic versus hierarchical modeling. , 2009, Journal of the American College of Surgeons.

[25]  S. Cole,et al.  Competing risk regression models for epidemiologic data. , 2009, American journal of epidemiology.

[26]  P. Austin,et al.  A Modification of the Elixhauser Comorbidity Measures Into a Point System for Hospital Death Using Administrative Data , 2009, Medical care.

[27]  David C Hoaglin,et al.  Modifying ICD-9-CM Coding of Secondary Diagnoses to Improve Risk-Adjustment of Inpatient Mortality Rates , 2009, Medical decision making : an international journal of the Society for Medical Decision Making.

[28]  Henrik Boström,et al.  Calibrating Random Forests , 2008, 2008 Seventh International Conference on Machine Learning and Applications.

[29]  Robin Genuer,et al.  Random Forests: some methodological insights , 2008, 0811.3619.

[30]  A. Mitchell,et al.  Why don’t patients attend their appointments? Maintaining engagement with psychiatric services , 2007 .

[31]  Kevin A. Clarke A Simple Distribution-Free Test for Nonnested Model Selection , 2007, Political Analysis.

[32]  A. Bottle,et al.  Intelligent information: a national system for monitoring clinical performance. , 2007, Health services research.

[33]  Azeem Majeed,et al.  Use of administrative data or clinical databases as predictors of risk of death in hospital: comparison of models , 2007, BMJ : British Medical Journal.

[34]  Vijay A. Singh,et al.  The Effects of Preexisting Medical Comorbidities on Mortality and Length of Hospital Stay in Acute Burn Injury: Evidence From a National Sample of 31,338 Adult Patients , 2007, Annals of surgery.

[35]  N. Cook Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction , 2007, Circulation.

[36]  Michael Green,et al.  Comparison between neural networks and multiple logistic regression to predict acute coronary syndrome in the emergency room , 2006, Artif. Intell. Medicine.

[37]  Azeem Majeed,et al.  Identifying patients at high risk of emergency hospital admissions: a logistic regression analysis. , 2006, Journal of the Royal Society of Medicine.

[38]  Mattias Ohlsson,et al.  Risk factor identification and mortality prediction in cardiac surgery using artificial neural networks. , 2006, The Journal of thoracic and cardiovascular surgery.

[39]  D. Wennberg,et al.  Case finding for patients at risk of readmission to hospital: development of algorithm to identify high risk patients , 2006, BMJ : British Medical Journal.

[40]  T. Osler,et al.  Impact of Changing the Statistical Methodology on Hospital and Surgeon Ranking: The Case of the New York State Cardiac Surgery Report Card , 2006, Medical care.

[41]  P. Kolh Importance of risk stratification models in cardiac surgery. , 2006, European heart journal.

[42]  C. Holman,et al.  A multipurpose comorbidity scoring system performed better than the Charlson index. , 2005, Journal of clinical epidemiology.

[43]  Peter J Pronovost,et al.  A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research. , 2005, Journal of critical care.

[44]  H. Quan,et al.  New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. , 2004, Journal of clinical epidemiology.

[45]  M. Bateson,et al.  Non-attendance at clinic: cycles of audit of a consultant based gastroenterology outpatient department , 2004, Postgraduate Medical Journal.

[46]  R. D'Agostino,et al.  Presentation of multivariate data for clinical use: The Framingham Study risk score functions , 2004, Statistics in medicine.

[47]  Peter C Austin,et al.  The Use of Fixed-and Random-Effects Models for Classifying Hospitals as Mortality Outliers: A Monte Carlo Assessment , 2003, Medical decision making : an international journal of the Society for Medical Decision Making.

[48]  K. Pearce,et al.  The development and validation of a comorbidity index for prostate cancer among Black men. , 2003, Journal of clinical epidemiology.

[49]  Albert W Wu,et al.  Noncardiac comorbidity increases preventable hospitalizations and mortality among Medicare beneficiaries with chronic heart failure. , 2003, Journal of the American College of Cardiology.

[50]  L. Bouter,et al.  How to measure comorbidity. a critical review of available methods. , 2003, Journal of clinical epidemiology.

[51]  Peter C Austin,et al.  A Comparison of Bayesian Methods for Profiling Hospital Performance , 2002, Medical decision making : an international journal of the Society for Medical Decision Making.

[52]  D. Sharp,et al.  Patient, hospital, and general practitioner characteristics associated with non-attendance: a cohort study. , 2002, The British journal of general practice : the journal of the Royal College of General Practitioners.

[53]  Christianna S. Williams,et al.  Development and Validation of a Risk‐Adjustment Index for Older Patients: The High‐Risk Diagnoses for the Elderly Scale , 2002, Journal of the American Geriatrics Society.

[54]  D. Sharp,et al.  Non-attendance at general practices and outpatient clinics , 2001, BMJ : British Medical Journal.

[55]  A. Hansell,et al.  Accessing and using hospital activity data. , 2001, Journal of public health medicine.

[56]  F. Dockery,et al.  The effect of reminder calls in reducing non-attendance rates at care of the elderly clinics , 2001, Postgraduate medical journal.

[57]  A. Hunter,et al.  Why patients do not attend for their appointments at a genetics clinic , 2000, Journal of medical genetics.

[58]  P. Hodgkin,et al.  Defaulters in general practice: who are they and what can be done about them? , 2000, Family practice.

[59]  M. Extermann,et al.  Measuring comorbidity in older cancer patients. , 2000, European journal of cancer.

[60]  A. Gatrad A completed audit to reduce hospital outpatients non-attendance rates , 2000, Archives of disease in childhood.

[61]  V. Devaraj,et al.  Reducing non-attendance at outpatient clinics , 1999, Journal of the Royal Society of Medicine.

[62]  R. Lancashire,et al.  Factors associated with clinic non‐attendance in adults with Type 1 diabetes mellitus , 1998, Diabetic medicine : a journal of the British Diabetic Association.

[63]  L. Bottaci,et al.  Artificial neural networks applied to outcome prediction for colorectal cancer patients in separate institutions , 1997, The Lancet.

[64]  D. Hosmer,et al.  A comparison of goodness-of-fit tests for the logistic regression model. , 1997, Statistics in medicine.

[65]  C. Morris,et al.  Development and validation of a claims based index for adjusting for risk of mortality: the case of acute myocardial infarction. , 1995, Journal of clinical epidemiology.

[66]  C. O'Brien,et al.  Factors affecting non-attendance in an ophthalmic outpatient department. , 1995, Journal of the Royal Society of Medicine.

[67]  M. Lloyd,et al.  Non-attendance at outpatient clinics: is it related to the referral process? , 1993, Family practice.

[68]  J. Morrow,et al.  Can Outpatient Non-Attendance Be Predicted from the Referral Letter? An Audit of Default at Neurology Clinics , 1991, Journal of the Royal Society of Medicine.

[69]  A. S. Mcneish,et al.  Understanding non-attendance in outpatient paediatric clinics. , 1990, Archives of disease in childhood.

[70]  A. Farrow,et al.  Non-attendance or non-invitation? A case-control study of failed outpatient appointments. , 1989, BMJ.

[71]  Q. Vuong Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses , 1989 .

[72]  M. Lynch,et al.  Lost to follow up: a study of nonattendance at a general paediatric outpatient clinic. , 1979, Archives of disease in childhood.

[73]  G. Schmidt,et al.  Applying competing risks regression models: an overview , 2012, Lifetime Data Analysis.

[74]  S Vijayakumar,et al.  The impact of age and comorbidity on survival outcomes and treatment patterns in prostate cancer , 2005, Prostate Cancer and Prostatic Diseases.

[75]  N. Santamaria,et al.  Why outpatients fail to attend their scheduled appointments: a prospective comparison of differences between attenders and non-attenders. , 2003, Australian health review : a publication of the Australian Hospital Association.

[76]  D. Bates,et al.  Using Diagnoses to Describe Populations and Predict Costs , 2000, Health care financing review.

[77]  C. Steiner,et al.  Comorbidity measures for use with administrative data. , 1998, Medical care.

[78]  C. Mackenzie,et al.  A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. , 1987, Journal of chronic diseases.