An approach to predicting patient experience through machine learning and social network analysis

Abstract Objective Improving the patient experience has become an essential component of any healthcare system’s performance metrics portfolio. In this study, we developed a machine learning model to predict a patient’s response to the Hospital Consumer Assessment of Healthcare Providers and Systems survey’s “Doctor Communications” domain questions while simultaneously identifying most impactful providers in a network. Materials and Methods This is an observational study of patients admitted to a single tertiary care hospital between 2016 and 2020. Using machine learning algorithms, electronic health record data were used to predict patient responses to Hospital Consumer Assessment of Healthcare Providers and Systems survey questions in the doctor domain, and patients who are at risk for responding negatively were identified. Model performance was assessed by area under receiver-operating characteristic curve. Social network analysis metrics were also used to identify providers most impactful to patient experience. Results Using a random forest algorithm, patients’ responses to the following 3 questions were predicted: “During this hospital stay how often did doctors. 1) treat you with courtesy and respect? 2) explain things in a way that you could understand? 3) listen carefully to you?” with areas under the receiver-operating characteristic curve of 0.876, 0.819, and 0.819, respectively. Social network analysis found that doctors with higher centrality appear to have an outsized influence on patient experience, as measured by rank in the random forest model in the doctor domain. Conclusions A machine learning algorithm identified patients at risk of a negative experience. Furthermore, a doctor social network framework provides metrics for identifying those providers that are most influential on the patient experience.

[1]  R. Holcombe,et al.  Demographic factors and hospital size predict patient satisfaction variance--implications for hospital value-based purchasing. , 2015, Journal of hospital medicine.

[2]  M. Stewart Effective physician-patient communication and health outcomes: a review. , 1995, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[3]  Cathal Doyle,et al.  A systematic review of evidence on the links between patient experience and clinical safety and effectiveness , 2013, BMJ Open.

[4]  Rui Jiang,et al.  A random forest approach to the detection of epistatic interactions in case-control studies , 2009, BMC Bioinformatics.

[5]  David G. Stork,et al.  Pattern Classification , 1973 .

[6]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[7]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[8]  Katrin Hambarsoomian,et al.  Do Hospitals Rank Differently on HCAHPS for Different Patient Subgroups? , 2010, Medical care research and review : MCRR.

[9]  S. Orr,et al.  Factors impacting Press Ganey patient satisfaction scores in orthopedic surgery spine clinic. , 2016, The spine journal : official journal of the North American Spine Society.

[10]  John W. Huppertz,et al.  Predicting HCAHPS scores from hospitals’ social media pages: A sentiment analysis , 2017, Health care management review.

[11]  DeWitt C Baldwin,et al.  How Residents Say They Learn: A National, Multi-Specialty Survey of First- and Second-Year Residents. , 2016, Journal of graduate medical education.

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  Denny Yu,et al.  Data-driven modeling of diabetes care teams using social network analysis , 2019, J. Am. Medical Informatics Assoc..

[14]  Takaya Saito,et al.  The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets , 2015, PloS one.

[15]  E. Zusman,et al.  HCAHPS replaces Press Ganey survey as quality measure for patient hospital experience. , 2012, Neurosurgery.

[16]  I. Press,et al.  Keys to patient satisfaction in the emergency department: results of a multiple facility study. , 1996, Hospital & health services administration.

[17]  A. Presson,et al.  Evidence of non-response bias in the Press-Ganey patient satisfaction survey , 2016, BMC Health Services Research.

[18]  E John Orav,et al.  The relation of patient satisfaction with complaints against physicians and malpractice lawsuits. , 2005, The American journal of medicine.

[19]  L. Markson,et al.  Discontinuation of use and switching of antidepressants: influence of patient-physician communication. , 2002, JAMA.

[20]  W. Katon,et al.  The patient-provider relationship: attachment theory and adherence to treatment in diabetes. , 2001, The American journal of psychiatry.

[21]  S. Younesian,et al.  Factors predicting patient satisfaction in the emergency department: a single-center study , 2017 .

[22]  Marc N Elliott,et al.  Development, Implementation, and Public Reporting of the HCAHPS Survey , 2010, Medical care research and review : MCRR.

[23]  J F Jekel,et al.  Perils, pitfalls, and possibilities in talking about medical risk. , 1999, JAMA.

[24]  Paul Sullivan,et al.  Diffusion of knowledge and behaviours among trainee doctors in an acute medical unit and implications for quality improvement work: a mixed methods social network analysis , 2019, BMJ Open.

[25]  William Olivero,et al.  Correlation Between Press Ganey Scores and Quality Outcomes From The National Neurosurgery Quality and Outcomes Database (Lumbar Spine) for a Hospital Employed Neurosurgical Practice. , 2018, Neurosurgery.

[26]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[27]  S. Edgman-Levitan,et al.  Analysis & commentary. Measuring patient experience as a strategy for improving primary care. , 2010, Health affairs.