ETECADx: Ensemble Self-Attention Transformer Encoder for Breast Cancer Diagnosis Using Full-Field Digital X-ray Breast Images

Early detection of breast cancer is an essential procedure to reduce the mortality rate among women. In this paper, a new AI-based computer-aided diagnosis (CAD) framework called ETECADx is proposed by fusing the benefits of both ensemble transfer learning of the convolutional neural networks as well as the self-attention mechanism of vision transformer encoder (ViT). The accurate and precious high-level deep features are generated via the backbone ensemble network, while the transformer encoder is used to diagnose the breast cancer probabilities in two approaches: Approach A (i.e., binary classification) and Approach B (i.e., multi-classification). To build the proposed CAD system, the benchmark public multi-class INbreast dataset is used. Meanwhile, private real breast cancer images are collected and annotated by expert radiologists to validate the prediction performance of the proposed ETECADx framework. The promising evaluation results are achieved using the INbreast mammograms with overall accuracies of 98.58% and 97.87% for the binary and multi-class approaches, respectively. Compared with the individual backbone networks, the proposed ensemble learning model improves the breast cancer prediction performance by 6.6% for binary and 4.6% for multi-class approaches. The proposed hybrid ETECADx shows further prediction improvement when the ViT-based ensemble backbone network is used by 8.1% and 6.2% for binary and multi-class diagnosis, respectively. For validation purposes using the real breast images, the proposed CAD system provides encouraging prediction accuracies of 97.16% for binary and 89.40% for multi-class approaches. The ETECADx has a capability to predict the breast lesions for a single mammogram in an average of 0.048 s. Such promising performance could be useful and helpful to assist the practical CAD framework applications providing a second supporting opinion of distinguishing various breast cancer malignancies.

[1]  Octavia A. Dobre,et al.  Federated Learning for Smart Healthcare: A Survey , 2021, ACM Comput. Surv..

[2]  S. Tummala,et al.  BreaST-Net: Multi-Class Classification of Breast Cancer from Histopathological Images Using Ensemble of Swin Transformers , 2022, Mathematics.

[3]  M. A. Al-masni,et al.  A Hybrid Workflow of Residual Convolutional Transformer Encoder for Breast Cancer Classification Using Digital X-ray Mammograms , 2022, Biomedicines.

[4]  M. A. Al-antari,et al.  BCNet: A Deep Learning Computer-Aided Diagnosis Framework for Human Peripheral Blood Cell Identification , 2022, Diagnostics.

[5]  M. A. Al-antari,et al.  Deep Learning Cascaded Feature Selection Framework for Breast Cancer Classification: Hybrid CNN with Univariate-Based Approach , 2022, Mathematics.

[6]  Wenjing Jian,et al.  Breast cancer detection and classification in mammogram using a three-stage deep learning framework based on PAA algorithm , 2022, Artif. Intell. Medicine.

[7]  R. K. Pateriya,et al.  A CNN-SVM based computer aided diagnosis of breast Cancer using histogram K-means segmentation technique , 2022, Multimedia Tools and Applications.

[8]  H. Nasiri,et al.  Classification of Breast Tumours Based on Histopathology Images Using Deep Features and Ensemble of Gradient Boosting Methods , 2022, ArXiv.

[9]  M. A. Al-antari,et al.  A hybrid explainable ensemble transformer encoder for pneumonia identification from chest X-ray images , 2022, Journal of advanced research.

[10]  I. Soerjomataram,et al.  Current and future burden of breast cancer: Global statistics for 2020 and 2040 , 2022, Breast.

[11]  Abdul Khader Jilani Saudagar,et al.  Breast Cancer Detection in Mammography Images Using Deep Convolutional Neural Networks and Fuzzy Ensemble Modeling Techniques , 2022, Diagnostics.

[12]  Y. Hou,et al.  A VGG attention vision transformer network for benign and malignant classification of breast ultrasound images. , 2022, Medical physics.

[13]  Wei Wang,et al.  Semi-supervised vision transformer with adaptive token sampling for breast cancer classification , 2022, Frontiers in Pharmacology.

[14]  H. Rivaz,et al.  Vision Transformers for Classification of Breast Ultrasound Images , 2022, 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[15]  Vidan F. Ghoneim,et al.  A Hybrid Deep Transfer Learning of CNN-Based LR-PCA for Breast Lesion Diagnosis via Medical Breast Mammograms , 2022, Sensors.

[16]  Y. Qiu,et al.  Transformers Improve Breast Cancer Diagnosis from Unregistered Multi-View Mammograms , 2022, Diagnostics.

[17]  Fayadh S. Alenezi,et al.  Deconv-transformer (DecT): A histopathological image classification model for breast cancer based on color deconvolution and transformer architecture , 2022, Information Sciences.

[18]  Samir B. Patel,et al.  Image Augmentation Techniques for Mammogram Analysis , 2022, J. Imaging.

[19]  H. Rajaguru,et al.  Multi-Deep CNN based Experimentations for Early Diagnosis of Breast Cancer , 2022, IETE Journal of Research.

[20]  F. Akhtar,et al.  Breast lesions classifications of mammographic images using a deep convolutional neural network-based approach , 2022, PloS one.

[21]  Jayesh George Melekoodappattu,et al.  Breast cancer detection in mammogram: combining modified CNN and texture feature based approach , 2022, Journal of Ambient Intelligence and Humanized Computing.

[22]  Paawan Sharma,et al.  Deep convolutional neural networks for computer-aided breast cancer diagnostic: a survey , 2022, Neural Computing and Applications.

[23]  Lars J. Grimm,et al.  Anomaly Detection of Calcifications in Mammography Based on 11,000 Negative Cases , 2021, IEEE Transactions on Biomedical Engineering.

[24]  Sumit K. Banchhor,et al.  Segmentation of malignant tumours in mammogram images: A hybrid approach using convolutional neural networks and connected component analysis , 2021, Expert Syst. J. Knowl. Eng..

[25]  D. Tao,et al.  A Survey on Vision Transformer , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Md. Zahid Hasan,et al.  BreastNet18: A High Accuracy Fine-Tuned VGG16 Model Evaluated Using Ablation Study for Diagnosing Breast Cancer from Enhanced Mammography Images , 2021, Biology.

[27]  Paawan Sharma,et al.  A Bottom-Up Review of Image Analysis Methods for Suspicious Region Detection in Mammograms , 2021, J. Imaging.

[28]  J. Dowling,et al.  A review of medical image data augmentation techniques for deep learning applications , 2021, Journal of medical imaging and radiation oncology.

[29]  Javad Rahebi,et al.  Multilevel thresholding of images with improved Otsu thresholding by black widow optimization algorithm , 2021, Multimedia Tools and Applications.

[30]  Holger Hermanns,et al.  What Do We Want From Explainable Artificial Intelligence (XAI)? - A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research , 2021, Artif. Intell..

[31]  A. Jemal,et al.  Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries , 2021, CA: a cancer journal for clinicians.

[32]  Qing-Long Zhang,et al.  SA-Net: Shuffle Attention for Deep Convolutional Neural Networks , 2021, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Stephen Marshall,et al.  A framework for breast cancer classification using Multi-DCNNs , 2021, Comput. Biol. Medicine.

[34]  Vasudevan Lakshminarayanan,et al.  DenseNet for Breast Tumor Classification in Mammographic Images , 2021, BIOMESIP.

[35]  H. Rajaguru,et al.  Automatic Detection and Classification of Mammograms Using Improved Extreme Learning Machine with Deep Learning , 2021 .

[36]  Mohammed Marey,et al.  YOLO Based Breast Masses Detection and Classification in Full-Field Digital Mammograms , 2020, Comput. Methods Programs Biomed..

[37]  S. M. Narangale,et al.  Breast Cancer Detection and Diagnosis Using Machine Learning: A Survey , 2021 .

[38]  S. Chavan,et al.  Breast Cancer Detection in Mammograms Using Deep Learning , 2021, Advances in Intelligent Systems and Computing.

[39]  Mohamed F. Tolba,et al.  Automated Breast Cancer Detection and Classification in Full Field Digital Mammograms Using Two Full and Cropped Detection Paths Approach , 2021, IEEE Access.

[40]  M. A. Al-antari,et al.  “Fast deep learning computer-aided diagnosis of COVID-19 based on digital chest x-ray images” , 2020, Applied Intelligence.

[41]  Mohamed F. Tolba,et al.  The Mass Size Effect on the Breast Cancer Detection Using 2-Levels of Evaluation , 2020, AISI.

[42]  Dilbag Singh,et al.  Classification of the COVID-19 infected patients using DenseNet201 based deep transfer learning , 2020, Journal of biomolecular structure & dynamics.

[43]  Tae-Seong Kim,et al.  Evaluation of deep learning detection and classification towards computer-aided diagnosis of breast lesions in digital X-ray mammograms , 2020, Comput. Methods Programs Biomed..

[44]  Hamed Tabrizchi,et al.  Breast cancer diagnosis using a multi-verse optimizer-based gradient boosting decision tree , 2020, SN Applied Sciences.

[45]  Mario Molinara,et al.  A multi-context CNN ensemble for small lesion detection , 2020, Artif. Intell. Medicine.

[46]  Ruey-Feng Chang,et al.  Computer-aided diagnosis of breast ultrasound images using ensemble learning from convolutional neural networks , 2020, Comput. Methods Programs Biomed..

[47]  Vinh Truong Hoang,et al.  Fusing of Deep Learning, Transfer Learning and GAN for Breast Cancer Histopathological Image Classification , 2019, ICCSAMA.

[48]  David Gutman,et al.  An Ensemble-based Active Learning for Breast Cancer Classification , 2019, 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[49]  M. Mutar,et al.  Pattern of Presentation of Patients With Breast Cancer in Iraq in 2018: A Cross-Sectional Study , 2019, Journal of global oncology.

[50]  Zhiwen Yu,et al.  A survey on ensemble learning , 2019, Frontiers of Computer Science.

[51]  Ji Ma,et al.  Dimension reduction of image deep feature using PCA , 2019, J. Vis. Commun. Image Represent..

[52]  Vinod Jagannath Kadam,et al.  Breast Cancer Diagnosis Using Feature Ensemble Learning Based on Stacked Sparse Autoencoders and Softmax Regression , 2019, Journal of Medical Systems.

[53]  Ali Idri,et al.  Reviewing ensemble classification methods in breast cancer , 2019, Comput. Methods Programs Biomed..

[54]  Tae-Seong Kim,et al.  A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification , 2018, Int. J. Medical Informatics.

[55]  Yasser M. Kadah,et al.  An Automatic Computer-Aided Diagnosis System for Breast Cancer in Digital Mammograms via Deep Belief Network , 2018 .

[56]  Edwin Valarezo,et al.  Simultaneous Detection and Classification of Breast Masses in Digital Mammograms via a Deep Learning YOLO-based CAD System , 2018, Comput. Methods Programs Biomed..

[57]  Rafik Goubran,et al.  Abnormality Detection in Mammography using Deep Convolutional Neural Networks , 2018, 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[58]  Li Shen,et al.  End-to-end Training for Whole Image Breast Cancer Diagnosis using An All Convolutional Design , 2017, ArXiv.

[59]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[60]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[62]  Nico Karssemeijer,et al.  Large scale deep learning for computer aided detection of mammographic lesions , 2017, Medical Image Anal..

[63]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Jaime S. Cardoso,et al.  INbreast: toward a full-field digital mammographic database. , 2012, Academic radiology.

[65]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[66]  N. Mathur,et al.  Biomonitoring of organochlorines in women with benign and malignant breast disease. , 2005, Environmental research.