Automated problem list generation and physicians perspective from a pilot study

OBJECTIVE An accurate, comprehensive and up-to-date problem list can help clinicians provide patient-centered care. Unfortunately, problem lists created and maintained in electronic health records by providers tend to be inaccurate, duplicative and out of date. With advances in machine learning and natural language processing, it is possible to automatically generate a problem list from the data in the EHR and keep it current. In this paper, we describe an automated problem list generation method and report on insights from a pilot study of physicians' assessment of the generated problem lists compared to existing providers-curated problem lists in an institution's EHR system. MATERIALS AND METHODS The natural language processing and machine learning-based Watson1 method models clinical thinking in identifying a patient's problem list using clinical notes and structured data. This pilot study assessed the Watson method and included 15 randomly selected, de-identified patient records from a large healthcare system that were each planned to be reviewed by at least two internal medicine physicians. The physicians created their own problem lists, and then evaluated the overall usefulness of their own problem lists (P), Watson generated problem lists (W), and the existing EHR problem lists (E) on a 10-point scale. The primary outcome was pairwise comparisons of P, W, and E. RESULTS Six out of the 10 invited physicians completed 27 assessments of P, W, and E, and in process evaluated 732 Watson generated problems and 444 problems in the EHR system. As expected, physicians rated their own lists, P, highest. However, W was rated higher than E. Among 89% of assessments, Watson identified at least one important problem that physicians missed. CONCLUSION Cognitive computing systems like this Watson system hold the potential for accurate, problem-list-centered summarization of patient records, potentially leading to increased efficiency, better clinical decision support, and improved quality of patient care.

[1]  Bharath Dandala,et al.  Scoring Disease-Medication Associations using Advanced NLP, Machine Learning, and Multiple Content Sources , 2016, BioTxtM@COLING 2016.

[2]  Peter J. Haug,et al.  Randomized controlled trial of an automated problem list with improved sensitivity , 2008, Int. J. Medical Informatics.

[3]  Erik T. Mueller,et al.  Watson: Beyond Jeopardy! , 2013, Artif. Intell..

[4]  Casey Holmes.  The problem list beyond meaningful use. Part I: The problems with problem lists. , 2011, Journal of AHIMA.

[5]  N. Mehta,et al.  Cognitive Computing for Electronic Medical Records , 2016 .

[6]  Christine A. Sinsky,et al.  Relationship Between Clerical Burden and Characteristics of the Electronic Environment With Physician Burnout and Professional Satisfaction. , 2016, Mayo Clinic proceedings.

[7]  Peter J. Haug,et al.  Improving the Sensitivity of the Problem List in an Intensive Care Unit by Using Natural Language Processing , 2006, AMIA.

[8]  Yoav Freund,et al.  The Alternating Decision Tree Learning Algorithm , 1999, ICML.

[9]  L. Weed.  Medical records that guide and teach. , 1968, The New England journal of medicine.

[10]  Peter J. Haug,et al.  Natural language processing to extract medical problems from electronic clinical documents: Performance evaluation , 2006, J. Biomed. Informatics.

[11]  Murthy V. Devarakonda,et al.  Problem-oriented patient record summary: An early report on a Watson application , 2014, 2014 IEEE 16th International Conference on e-Health Networking, Applications and Services (Healthcom).

[12]  Dean F Sittig,et al.  The Burden of Inbox Notifications in Commercial Electronic Health Records. , 2016, JAMA internal medicine.

[13]  Murthy V. Devarakonda,et al.  Automated Problem List Generation from Electronic Medical Records in IBM Watson , 2015, AAAI.

[14]  S Velupillai,et al.  Recent Advances in Clinical Natural Language Processing in Support of Semantic Analysis , 2015, Yearbook of Medical Informatics.

[15]  Peter J. Haug,et al.  Comparing Natural Language Processing Tools to Extract Medical Problems from Narrative Text , 2005, AMIA.

[16]  Robert M. Wachter,et al.  The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine's Computer Age , 2015 .

[17]  Murthy V. Devarakonda,et al.  Ground Truth Creation for Complex Clinical NLP Tasks – an Iterative Vetting Approach and Lessons Learned , 2017, CRI.

[18]  Noémie Elhadad,et al.  Automated methods for the summarization of electronic health records , 2015, J. Am. Medical Informatics Assoc..

[19]  Sunghwan Sohn,et al.  Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications , 2010, J. Am. Medical Informatics Assoc..