Automatic Caption Generation of Retinal Diseases with Self-trained RNN Merge Model

Retinal diseases may lead to blindness. So, early detection of the diseases may prevent the scenario. Automatic caption generation from the retinal images is the challenging method to diagnose the diseases. In the recent era, deep learning is very popular in analyzing the medical images effectively. In this paper, we propose an automatic caption generation method based on CNN and self-trained bidirectional LSTM model. We build the model based on merge architecture. The model is trained with normal and abnormal retinal images and their labeled diseases. The target is to generate the caption of true disease from a given test image. We use the STARE database to build the model. After the experiment, it can be said that our attempts are promising and experimental result shows that the model performs well in generating the caption from the new given retinal image. We also evaluate the score using BLEU metrics.

[1]  Yanchun Liang,et al.  Image Captioning with Bidirectional Semantic Attention-Based Guiding of Long Short-Term Memory , 2019, Neural Processing Letters.

[2]  Björn W. Schuller,et al.  Contextual Bidirectional Long Short-Term Memory Recurrent Neural Network Language Models: A Generative Approach to Sentiment Analysis , 2017, EACL.

[3]  Ronald M. Summers,et al.  Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Changshui Zhang,et al.  Aligning where to see and what to tell: image caption with region-based attention and scene factorization , 2015, ArXiv.

[6]  Yejin Choi,et al.  Baby talk: Understanding and generating simple image descriptions , 2011, CVPR 2011.

[7]  Jiebo Luo,et al.  Image Captioning with Semantic Attention , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Pengtao Xie,et al.  On the Automatic Generation of Medical Imaging Reports , 2017, ACL.

[9]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[10]  Tien Yin Wong,et al.  Glaucoma detection based on deep convolutional neural network , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[11]  Xu Jia,et al.  Guiding the Long-Short Term Memory Model for Image Caption Generation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Imane Allaouzi,et al.  Automatic Caption Generation for Medical Images , 2018, SCA.

[13]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[14]  Wojciech Zaremba,et al.  An Empirical Exploration of Recurrent Network Architectures , 2015, ICML.

[15]  Albert Gatt,et al.  What is the Role of Recurrent Neural Networks (RNNs) in an Image Caption Generator? , 2017, INLG.

[16]  Subhransu Maji,et al.  Automatic Image Annotation using Deep Learning Representations , 2015, ICMR.