Automatic population of structured reports from narrative pathology reports

The aim of this project is to use the methods of natural language processing to extract pertinent information from free-text pathology reports to automatically populate structured reports. A processing pipeline has been developed cosseting of a combination of a supervised machine learning based approach using Conditional Random Fields for medical entity recognition and some rule-based methods. In total 477 narrative pathology reports of primary cutaneous melanomas were collected for evaluation. Evaluations on the training set show that system performance can be improved by about 8.7% by refinement of the rules. The overall micro-averaged precision, recall and F-score of end to end evaluation on the test set are 89.44%, 80.60% and 84.79% respectively. Our study indicates the feasibility of this approach to automate the population of structured template from narrative reports with promising results. Error analysis reveals that a single specimen report with standard headings and the presence of simple and concise statements is significantly associated with correct populations. In conclusion, the system can improve pathology reporting, and data mining for cancer registries, clinical audits and epidemiology research.