BreakHis based breast cancer automatic diagnosis using deep learning: Taxonomy, survey and insights

Abstract There are several breast cancer datasets for building Computer Aided Diagnosis systems (CADs) using either deep learning or traditional models. However, most of these datasets impose various trade-offs on practitioners related to their availability or inner clinical value. Recently, a public dataset called BreakHis has been released to overcome these limitations. BreakHis is organized into four magnification levels, each image is labeled according to its main category (Benign/Malignant) and its subcategory (A/F/PT/TA/PC/DC/LC/MC). This organization allows practitioners to address this problem either as a binary or a multi-category classification task with either a magnification dependent or independent training approach. In this work, we define a taxonomy that categorize this problem into four different reformulations: Magnification-Specific Binary (MSB), Magnification-Independent Binary (MIB), Magnification-Specific Multi-category (MSM) and Magnification-Independent Multi-category (MIM) classifications. We provide a comprehensive survey of all related works. We identify the best reformulation from clinical and practical standpoints. Finally, we explore for the first time the MIM approach using deep learning and draw the learnt lessons.

[1]  Luiz Eduardo Soares de Oliveira,et al.  Multiple instance learning for histopathological breast cancer image classification , 2019, Expert Syst. Appl..

[2]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  C. Mathers,et al.  Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012 , 2015, International journal of cancer.

[4]  Caglar Senaras,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[5]  K. Narasimhan,et al.  Automated Diagnosis of Breast Cancer Using Wavelet Based Entropy Features , 2018 .

[6]  Xiangjian He,et al.  Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges , 2019, Journal of Digital Imaging.

[7]  Shin-Jye Lee,et al.  Image Classification Based on the Boost Convolutional Neural Network , 2018, IEEE Access.

[8]  Eric P. Xing,et al.  Structured Literature Image Finder: Extracting Information from Text and Images in Biomedical Literature , 2009, BioLINK@ISMB/ECCB.

[9]  Baolin Du,et al.  Breast Cancer Histopathological Image Classification via Deep Active Learning and Confidence Boosting , 2018, ICANN.

[10]  Emmanuel Adetiba,et al.  Medical Image Classification with Hand-Designed or Machine-Designed Texture Descriptors: A Performance Evaluation , 2018, IWBBIO.

[11]  Dominique Zosso,et al.  Variational Mode Decomposition , 2014, IEEE Transactions on Signal Processing.

[12]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[13]  Jack A Tuszynski,et al.  Automatic prediction of tumour malignancy in breast cancer with fractal dimension , 2016, Royal Society Open Science.

[14]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[15]  Qiu-Hua Lin,et al.  Machine Learning for Medical Imaging , 2019, Journal of healthcare engineering.

[16]  Luca Maria Gambardella,et al.  Assessment of algorithms for mitosis detection in breast cancer histopathology images , 2014, Medical Image Anal..

[17]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[18]  Vincent Lepetit,et al.  Keypoint recognition using randomized trees , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Luiz Eduardo Soares de Oliveira,et al.  Breast cancer histopathological image classification using Convolutional Neural Networks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[20]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Eunjeong Park,et al.  A method for classifying medical images using transfer learning: A pilot study on histopathology of breast cancer , 2017, 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (Healthcom).

[22]  Juho Kannala,et al.  Deep learning for magnification independent breast cancer histopathology image classification , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[23]  Yilong Yin,et al.  Deep learning model based breast cancer histopathological image classification , 2017, 2017 IEEE 2nd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA).

[24]  Ümit Budak,et al.  Transfer learning based histopathologic image classification for breast cancer detection , 2018, Health Information Science and Systems.

[25]  Mei Chen,et al.  Low Dimensional Representation of Fisher Vectors for Microscopy Image Classification , 2017, IEEE Transactions on Medical Imaging.

[26]  Arnav Bhavsar,et al.  Automated Classification for Breast Cancer Histopathology Images: Is Stain Normalization Important? , 2017, CARE/CLIP@MICCAI.

[27]  Shihui Ying,et al.  Histopathological image classification with bilinear convolutional neural networks , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[28]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[29]  Sunghwan Sohn,et al.  Deep learning and alternative learning strategies for retrospective real-world clinical data , 2019, npj Digital Medicine.

[30]  Jasjit S Suri,et al.  State-of-the-art review on deep learning in medical imaging. , 2019, Frontiers in bioscience.

[31]  Arnav Bhavsar,et al.  Breast Cancer Histopathological Image Classification: Is Magnification Important? , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  M. A. Aswathy,et al.  Detection of breast cancer on digital histopathology images: Present status and future possibilities , 2017 .

[33]  Kai Zhang,et al.  Deep learning for image-based cancer detection and diagnosis - A survey , 2018, Pattern Recognit..

[34]  Andrew Evans,et al.  Digital imaging in pathology: whole-slide imaging and beyond. , 2013, Annual review of pathology.

[35]  Jianzhong Wu,et al.  Stacked Sparse Autoencoder (SSAE) based framework for nuclei patch classification on breast cancer histopathology , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[36]  Dragica Radosav,et al.  Deep Learning and Medical Diagnosis: A Review of Literature , 2018, Multimodal Technol. Interact..

[37]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[38]  Yuanjie Zheng,et al.  Breast Cancer Multi-classification from Histopathological Images with Structured Deep Learning Model , 2017, Scientific Reports.

[39]  M. L. Fravolini,et al.  Dimensionality Reduction Strategies for CNN-Based Classification of Histopathological Images , 2018, IIMSS.

[40]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[41]  A. Sahin,et al.  Benign breast diseases: classification, diagnosis, and management. , 2006, The oncologist.

[42]  Shutao Li,et al.  Gene Selection Using Wilcoxon Rank Sum Test and Support Vector Machine for Cancer Classification , 2007, CIS.

[43]  Thomas Mensink,et al.  Improving the Fisher Kernel for Large-Scale Image Classification , 2010, ECCV.

[44]  Yinan Kong,et al.  Histopathological Breast-Image Classification Using Concatenated R–G–B Histogram Information , 2019 .

[45]  Saso Dzeroski,et al.  Combining Classifiers with Meta Decision Trees , 2003, Machine Learning.

[46]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[47]  Abhijit Guha Roy,et al.  Classifying histopathology whole-slides using fusion of decisions from deep convolutional network on a collection of random multi-views at multi-magnification , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[48]  Anne L. Martel,et al.  Transitioning Between Convolutional and Fully Connected Layers in Neural Networks , 2017, DLMIA/ML-CDS@MICCAI.

[49]  Tony X. Han,et al.  Multiple Instance Learning Convolutional Neural Networks for object recognition , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[50]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[51]  Alexander Binder,et al.  Comparison of deep learning architectures for H&E histopathology images , 2017, 2017 IEEE Conference on Big Data and Analytics (ICBDA).

[52]  Alaa Tharwat,et al.  Linear vs. quadratic discriminant analysis classifier: a tutorial , 2016, Int. J. Appl. Pattern Recognit..

[53]  Catarina Eloy,et al.  Classification of breast cancer histology images using Convolutional Neural Networks , 2017, PloS one.

[54]  Ju Jia Zou,et al.  Adapting fisher vectors for histopathology image classification , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[55]  Heikki Mannila,et al.  Random projection in dimensionality reduction: applications to image and text data , 2001, KDD '01.

[56]  Mark J. Shensa,et al.  The discrete wavelet transform: wedding the a trous and Mallat algorithms , 1992, IEEE Trans. Signal Process..

[57]  Zhenghao Chen,et al.  Using Transfer Learning with Convolutional Neural Networks to Diagnose Breast Cancer from Histopathological Images , 2017, ICONIP.

[58]  Mohammad Faizal Ahmad Fauzi,et al.  Classification of benign and malignant tumors in histopathology images , 2017, 2017 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[59]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[60]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[61]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Zhiguo Jiang,et al.  Size-Scalable Content-Based Histopathological Image Retrieval From Database That Consists of WSIs , 2018, IEEE Journal of Biomedical and Health Informatics.

[63]  Abhijit Guha Roy,et al.  Multiple instance learning of deep convolutional neural networks for breast histopathology whole slide classification , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[64]  Arnav Bhavsar,et al.  An Integrated Multi-scale Model for Breast Cancer Histopathological Image Classification with Joint Colour-Texture Features , 2017, CAIP.

[65]  Hong Zhang,et al.  Facial expression recognition via learning deep sparse autoencoders , 2018, Neurocomputing.

[66]  Christian Riess,et al.  A Gentle Introduction to Deep Learning in Medical Image Processing , 2018, Zeitschrift fur medizinische Physik.

[67]  Ainuddin Wahid Abdul Wahab,et al.  Deep learning-based breast cancer classification through medical imaging modalities: state of the art and research challenges , 2019, Artificial Intelligence Review.

[68]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[69]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[70]  Yinan Kong,et al.  Histopathological Breast-Image Classification Using Local and Frequency Domains by Convolutional Neural Network , 2018, Inf..

[71]  Stan Szpakowicz,et al.  Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation , 2006, Australian Conference on Artificial Intelligence.

[72]  Lilly Suriani Affendey,et al.  Classification of Histopathology Images of Breast into Benign and Malignant using a Single-layer Convolutional Neural Network , 2017, ICISPC 2017.

[73]  Ziba Gandomkar,et al.  MuDeRN: Multi-category classification of breast histopathological image using deep residual networks , 2018, Artif. Intell. Medicine.

[74]  Mahua Bhattacharya,et al.  Classification of breast tumors as benign and malignant using textural feature descriptor , 2017, 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[75]  Mateusz Buda,et al.  Deep learning in radiology: An overview of the concepts and a survey of the state of the art with focus on MRI , 2018, Journal of magnetic resonance imaging : JMRI.

[76]  Luiz Eduardo Soares de Oliveira,et al.  Deep features for breast cancer histopathological image classification , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[77]  Krzysztof Michalak,et al.  Correlation based feature selection method , 2010, Int. J. Bio Inspired Comput..

[78]  Fuad E. Alsaadi,et al.  Deep Belief Networks for Quantitative Analysis of a Gold Immunochromatographic Strip , 2016, Cognitive Computation.

[79]  Jesús González,et al.  Classification of Breast Cancer Histopathological Images Using KAZE Features , 2018, IWBBIO.

[80]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[81]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[82]  Pendar Alirezazadeh,et al.  Representation learning-based unsupervised domain adaptation for classification of breast cancer histopathology images , 2018 .

[83]  Yinan Kong,et al.  Histopathological breast-image classification with restricted Boltzmann machine along with backpropagation , 2018 .

[84]  Kundan Kumar,et al.  Breast cancer classification of image using convolutional neural network , 2018, 2018 4th International Conference on Recent Advances in Information Technology (RAIT).

[85]  Arnav Bhavsar,et al.  Sequential Modeling of Deep Features for Breast Cancer Histopathological Image Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[86]  Yinan Kong,et al.  Histopathological Breast Cancer Image Classification by Deep Neural Network Techniques Guided by Local Clustering , 2018, BioMed research international.

[87]  Francisco Herrera,et al.  A first study exploring the performance of the state-of-the art CNN model in the problem of breast cancer , 2018, LOPAL '18.

[88]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[89]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[90]  Lei Wang,et al.  HEp-2 Cell Image Classification With Deep Convolutional Neural Networks , 2015, IEEE Journal of Biomedical and Health Informatics.

[91]  Subhransu Maji,et al.  Deep filter banks for texture recognition and segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[92]  Ming Xiao,et al.  Histopathological Image Recognition with Domain Knowledge Based Deep Features , 2018, ICIC.

[93]  Heng Huang,et al.  Supervised Intra-embedding of Fisher Vectors for Histopathology Image Classification , 2017, MICCAI.

[94]  Markus Ringnér,et al.  What is principal component analysis? , 2008, Nature Biotechnology.

[95]  Lilly Suriani Affendey,et al.  Transferred Semantic Scores for Scalable Retrieval of Histopathological Breast Cancer Images , 2018, International Journal of Multimedia Information Retrieval.

[96]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[97]  Max A. Viergever,et al.  Breast Cancer Histopathology Image Analysis: A Review , 2014, IEEE Transactions on Biomedical Engineering.

[98]  Baoxin Li,et al.  Simpler Non-Parametric Methods Provide as Good or Better Results to Multiple-Instance Learning , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[99]  Yang Gao,et al.  Feature learning with component selective encoding for histopathology image classification , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[100]  Luiz Eduardo Soares de Oliveira,et al.  A Dataset for Breast Cancer Histopathological Image Classification , 2016, IEEE Transactions on Biomedical Engineering.

[101]  Benoit B. Mandelbrot,et al.  Fractal Geometry of Nature , 1984 .

[102]  Kun Zhang,et al.  Classification of Breast Cancer Based on Histology Images Using Convolutional Neural Networks , 2018, IEEE Access.

[103]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[104]  Max Welling,et al.  Group Equivariant Convolutional Networks , 2016, ICML.

[105]  O. Metzger-Filho,et al.  Differences between invasive lobular and invasive ductal carcinoma of the breast: results and therapeutic implications , 2016, Therapeutic advances in medical oncology.

[106]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.