Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images

Convolutional neural networks (CNNs) are deep learning network architectures that have pushed forward the state-of-the-art in a range of computer vision applications and are increasingly popular in medical image analysis. However, training of CNNs is time-consuming and challenging. In medical image analysis tasks, the majority of training examples are easy to classify and therefore contribute little to the CNN learning process. In this paper, we propose a method to improve and speed-up the CNN training for medical image analysis tasks by dynamically selecting misclassified negative samples during training. Training samples are heuristically sampled based on classification by the current status of the CNN. Weights are assigned to the training samples and informative samples are more likely to be included in the next CNN training iteration. We evaluated and compared our proposed method by training a CNN with (SeS) and without (NSeS) the selective sampling method. We focus on the detection of hemorrhages in color fundus images. A decreased training time from 170 epochs to 60 epochs with an increased performance-on par with two human experts-was achieved with areas under the receiver operating characteristics curve of 0.894 and 0.972 on two data sets. The SeS CNN statistically outperformed the NSeS CNN on an independent test set.

[1]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[2]  P. Aimmanee,et al.  A survey on hemorrhage detection in diabetic retinopathy retinal images , 2012, 2012 9th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology.

[3]  C. Lee Giles,et al.  Learning on the border: active learning in imbalanced data classification , 2007, CIKM '07.

[4]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[6]  Miss. S. Deepa,et al.  Certain Investigation of the Retinal Hemorrhage Detection in Fundus Images , 2015 .

[7]  Fabio A. González,et al.  A Deep Learning Architecture for Image Representation, Visual Interpretability and Automated Basal-Cell Carcinoma Cancer Detection , 2013, MICCAI.

[8]  H. Sebastian Seung,et al.  Selective Sampling Using the Query by Committee Algorithm , 1997, Machine Learning.

[9]  Tien Yin Wong,et al.  Diabetic retinopathy , 2010, The Lancet.

[10]  Bálint Antal,et al.  Improving microaneurysm detection in color fundus images by using context-aware approaches , 2013, Comput. Medical Imaging Graph..

[11]  Xin Yao,et al.  Dynamic Sampling Approach to Training Neural Networks for Multiclass Imbalance Classification , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[13]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Hao Chen,et al.  Automatic detection of cerebral microbleeds via deep learning based 3D feature representation , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[15]  C. Sinthanayothin,et al.  Automated detection of diabetic retinopathy on digital fundus images , 2002, Diabetic medicine : a journal of the British Diabetic Association.

[16]  M. Usman Akram,et al.  Automated Detection of Dark and Bright Lesions in Retinal Images for Early Detection of Diabetic Retinopathy , 2012, Journal of Medical Systems.

[17]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[18]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[19]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[20]  Bram van Ginneken,et al.  Automatic detection of red lesions in digital color fundus photographs , 2005, IEEE Transactions on Medical Imaging.

[21]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[22]  Frank W. Samuelson,et al.  Comparing image detection algorithms using resampling , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[23]  Hadi Seyedarabi,et al.  A Comparative Study on Preprocessing Techniques in Diabetic Retinopathy Retinal Images: Illumination Correction and Contrast Enhancement , 2015, Journal of medical signals and sensors.

[24]  Giri Babu Kande,et al.  Automatic Detection of Microaneurysms and Hemorrhages in Digital Fundus Images , 2010, Journal of Digital Imaging.

[25]  Xin He,et al.  ROC, LROC, FROC, AFROC: an alphabet soup. , 2009, Journal of the American College of Radiology : JACR.

[26]  Nico Karssemeijer,et al.  Automated localization of breast cancer in DCE-MRI , 2015, Medical Image Anal..

[27]  Nico Karssemeijer,et al.  Computer-aided detection versus independent double reading of masses on mammograms. , 2003, Radiology.

[28]  Peter Norvig,et al.  The Unreasonable Effectiveness of Data , 2009, IEEE Intelligent Systems.

[29]  Ronald M. Summers,et al.  2D View Aggregation for Lymph Node Detection Using a Shallow Hierarchy of Linear Classifiers , 2014, MICCAI.

[30]  Gwénolé Quellec,et al.  Optimal Wavelet Transform for the Detection of Microaneurysms in Retina Photographs , 2008, IEEE Transactions on Medical Imaging.

[31]  Joseph M. Reinhardt,et al.  Splat Feature Classification With Application to Retinal Hemorrhage Detection in Fundus Images , 2013, IEEE Transactions on Medical Imaging.

[32]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[33]  Pavlos S. Efraimidis,et al.  Weighted Random Sampling , 2008, Encyclopedia of Algorithms.

[34]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[35]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[36]  M. Sonka,et al.  Retinal Imaging and Image Analysis , 2010, IEEE Reviews in Biomedical Engineering.

[37]  P. Gupta,et al.  An Efficient Parallel Algorithm for Random Sampling , 1989, Inf. Process. Lett..

[38]  Adam Lipowski,et al.  Roulette-wheel selection via stochastic acceptance , 2011, ArXiv.

[39]  Dinggang Shen,et al.  Segmenting Hippocampus from Infant Brains by Sparse Patch Matching with Deep-Learned Features , 2014, MICCAI.

[40]  Andrés G. Marrugo,et al.  Retinal image analysis: preprocessing and feature extraction , 2011 .

[41]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[42]  Nitesh V. Chawla,et al.  Editorial: special issue on learning from imbalanced data sets , 2004, SKDD.

[43]  Thomas Theelen,et al.  Automatic drusen quantification and risk assessment of age-related macular degeneration on color fundus images. , 2013, Investigative ophthalmology & visual science.

[44]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[45]  Frank W. Samuelson,et al.  ADVANTAGES AND EXAMPLES OF RESAMPLING FOR CAD EVALUATION , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[46]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..