A computer-aided diagnosis system for breast cancer molecular subtype prediction in mammographic images

Abstract Several computer-aided diagnosis (CAD) systems have been developed to assist the radiologists for early breast cancer detection and treatment. These CAD systems provide statistical features of a mammogram using computer vision and image processing techniques for characterizing the morphological structure and evolution of the tumors. In this chapter a CAD system is introduced that includes three stages: tumor detection, segmentation, and tumor-shape and molecular subtypes classification based on deep learning models. The first stage is to detect the region of interest (ROI) that contains a tumor from mammographic images by using a modified Faster R-CNN (convolutional neural network) detector, which incorporates an Inception-ResNet-v2 feature extractor with a squeeze and excitation network. While the second stage employs a conditional generative adversarial network (cGAN) to segment the breast tumor from the detected ROI. For shape classification, a CNN is then developed in the third stage of the CAD system to classify the binary masks of the cGAN network into four tumor-shape classes: irregular, lobular, oval, and round. Finally, this chapter presents a study of the correlation between the tumor shapes and molecular subtypes of breast cancer. The findings of this chapter indicate that the tumor shape can be analyzed for understanding the molecular subtype of the tumor.