Real-time Machine Learning Alerts to Prevent Escalation of Care: A Pragmatic Clinical Trial

Importance: Automated machine learning algorithms have been shown to outperform older methods in predicting clinical deterioration requiring escalation of care, but rigorous prospective data on their real-world efficacy are limited. Objective: We hypothesized that real-time deterioration prediction alerts sent directly to front-line providers would reduce escalations. Design: Single-center prospective pragmatic clinical trial conducted from July 2019 to March 2020. The trial was terminated early due to the COVID-19 pandemic. Patients were followed for 30 days post-discharge. Setting: Academic tertiary care medical center located in New York City. Participants: All adult patients admitted to any of four medical-surgical units were included. Assignment to intervention or control arms was determined by initial unit admission. Intervention: Real-time alerts stratified according to predicted likelihood of clinical deterioration sent to the nursing/primary team or directly to the rapid response team. Clinical care and interventions were at the discretion of the providers. For the control units, alerts were generated but not sent. Main Outcomes: The primary outcome was the incidence of escalation of care. Secondary outcomes included orders placed for cardiovascular support, in-hospital and 30-day mortality. Ad-hoc outcomes included time to ICU escalation and time to discharge. Results: 2,780 patients were enrolled, 1,506 in the intervention group and the 1,274 in the control group. Average age was 66.2 years and 1,446 (52%) of participants were female. There was no difference in escalation between the groups, relative risk(RR) 1.22(95% Confidence Interval[CI] (0.97,1.54),p=0.10). Patients in the intervention group were more likely to receive cardiovascular support orders RR 1.35(95% CI (1.10,1.66),p=0.022). Median time to escalation with alerts was 50.6 [21.6-103] versus 58.6 [25.4-115] hours (difference -5.70;95% CI (-10.00,-2.00),p<0.001). The hazard ratio for likelihood of ICU escalation within 12 hours of an alert was 3.36 (95% CI (1.38,8.21),p=0.01) and for faster hospital discharge was 1.10 (95% CI (1.01,1.19),p=0.02). Combined in-hospital and 30-day-mortality was lower in the intervention group, RR 0.72 (95% CI (0.54,0.94),p=0.01). Conclusions and Relevance: Preliminary evidence suggests that real-time machine learning alerts do not reduce the incidence of escalation but are effective in reducing time to escalation, hospital length of stay and mortality. Trial Registration: ClinicalTrials.gov, NCT04026555, https://clinicaltrials.gov