A Pathology Image Diagnosis Network with Visual Interpretability and Structured Diagnostic Report

Despite recent advances in medical diagnosis domain, many challenges remain in obtaining more accurate conclusions and in presenting semantically and visually interpretable results during the diagnosis process. An interpretable diagnosis process is proposed through the implementation of a deep learning model. This consists of three interrelated models, an image model, an attention model and a conclusion model. The proposed image model extracts the semantic feature using convolutional neural networks (CNNs). The conclusion model, integrated with the semantic attributes attention model, aims to predict the conclusion label by long-short term memory (LSTM), which captures the discriminative relationship between semantic attributes. The network is trained in end-to-end way with different weight of each model. Based upon a cervical intraepithelial neoplasia images, diagnostic report and labels (CINDRAL) dataset, the approach demonstrates significant improvement when comparing the baseline in the conclusion result.