Towards Developing an Intelligent Agent to Assist in Patient Diagnosis Using Neural Networks on Unstructured Patient Clinical Notes: Initial Analysis and Models

Abstract Technological advances in information-communication technologies in the health ecosystem have allowed for the recording and consumption of massive amounts of structured and unstructured health data. In developing countries, the use of Electronic Medical Records (EMR) is necessary to address the need for efficient delivery of services and informed decision-making, especially at the local level where health facilities and practitioners may be lacking. Text mining is a variation of data mining that tries to extract non-trivial information and knowledge from unstructured text. This study aims to determine the feasibility of integrating an intelligent agent within EMRs for automatic diagnosis prediction based on the unstructured clinical notes. A Multilayer Feed-Forward Neural Network with Back Propagation training was implemented for classification. The two neural network models predicted hypertension against similar diagnoses with 11.52% and 10.53% percent errors but predicted with 54.01% and 64.82% percent errors when used on a group of similar diagnoses. Further development is needed for prediction of diagnoses with common symptoms and related diagnoses. The results still prove, however, that unstructured data possesses value beneficial for clinical decision support. If further analyzed with structured data, a more accurate intelligent agent may be explored.