Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning

OBJECTIVE The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics. MATERIALS AND METHODS We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit. RESULTS We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57 234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness. CONCLUSIONS The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.

[1]  Mayur B. Patel,et al.  Network Analysis Subtleties in ICU Structures and Outcomes , 2020, American journal of respiratory and critical care medicine.

[2]  You Chen,et al.  Metrics for assessing physician activity using electronic health record log data , 2020, J. Am. Medical Informatics Assoc..

[3]  J. Overhage,et al.  Physician Time Spent Using the Electronic Health Record During Outpatient Encounters , 2020, Annals of Internal Medicine.

[4]  Vimla L. Patel,et al.  EHR audit logs: A new goldmine for health services research? , 2019, J. Biomed. Informatics.

[5]  Michelle R. Hribar,et al.  Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods , 2019, J. Am. Medical Informatics Assoc..

[6]  You Chen,et al.  Modeling Care Team Structures in the Neonatal Intensive Care Unit through Network Analysis of EHR Audit Logs , 2019, Methods of Information in Medicine.

[7]  Josef Spidlen,et al.  Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets , 2019, Nature Communications.

[8]  Philip J. Kroth,et al.  Association of Electronic Health Record Design and Use Factors With Clinician Stress and Burnout , 2019, JAMA network open.

[9]  Genna R. Cohen,et al.  Variation in Physicians’ Electronic Health Record Documentation and Potential Patient Harm from That Variation , 2019, Journal of General Internal Medicine.

[10]  Hude Quan,et al.  Evaluation of interventions to improve electronic health record documentation within the inpatient setting: a protocol for a systematic review , 2019, Systematic Reviews.

[11]  Vincent A. Traag,et al.  From Louvain to Leiden: guaranteeing well-connected communities , 2018, Scientific Reports.

[12]  Bradley Malin,et al.  Interaction patterns of trauma providers are associated with length of stay , 2018, J. Am. Medical Informatics Assoc..

[13]  Alan J. Card Physician Burnout: Resilience Training is Only Part of the Solution , 2018, The Annals of Family Medicine.

[14]  Fabian J Theis,et al.  SCANPY: large-scale single-cell gene expression data analysis , 2018, Genome Biology.

[15]  Brian G. Arndt,et al.  Tethered to the EHR: Primary Care Physician Workload Assessment Using EHR Event Log Data and Time-Motion Observations , 2017, The Annals of Family Medicine.

[16]  B. Reeder,et al.  Use of Electronic Health Records by Nurses for Symptom Management in Inpatient Settings: A Systematic Review , 2017, Computers, informatics, nursing : CIN.

[17]  A. Reid,et al.  Electronic Health Record Effects on Work-Life Balance and Burnout Within the I3 Population Collaborative. , 2017, Journal of graduate medical education.

[18]  Uta S Guo,et al.  Racing Against the Clock: Internal Medicine Residents' Time Spent On Electronic Health Records. , 2016, Journal of graduate medical education.

[19]  Bradley Malin,et al.  We work with them? Healthcare workers interpretation of organizational relations mined from electronic health records , 2014, Int. J. Medical Informatics.

[20]  E. Hess,et al.  Electronic medical records and physician stress in primary care: results from the MEMO Study. , 2014, Journal of the American Medical Informatics Association : JAMIA.

[21]  Julia Adler-Milstein,et al.  The impact of electronic health record use on physician productivity. , 2013, The American journal of managed care.

[22]  Luís Velez Lapão,et al.  Analysis of the quality of hospital information systems audit trails , 2013, BMC Medical Informatics and Decision Making.

[23]  David E. Irwin,et al.  Finding a "Kneedle" in a Haystack: Detecting Knee Points in System Behavior , 2011, 2011 31st International Conference on Distributed Computing Systems Workshops.

[24]  Wil M. P. van der Aalst,et al.  Decision Mining in ProM , 2006, Business Process Management.

[25]  G. Ruxton The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test , 2006 .

[26]  K. Kim,et al.  Face recognition using kernel principal component analysis , 2002, IEEE Signal Process. Lett..

[27]  Martin F. Arlitt,et al.  Characterizing Web user sessions , 2000, PERV.

[28]  Rachel L. Ross,et al.  Burnout and EHR use among academic primary care physicians with varied clinical workloads. , 2019, AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science.

[29]  Kevin B. Johnson,et al.  Breadcrumbs: Assessing the Feasibility of Automating Provider Documentation Using Electronic Health Record Activity , 2018, AMIA.

[30]  He Zhang,et al.  Inferring Clinical Workflow Efficiency via Electronic Medical Record Utilization , 2015, AMIA.

[31]  Juan Enrique Ramos,et al.  Using TF-IDF to Determine Word Relevance in Document Queries , 2003 .