The purpose of this study was to develop a deep learning model to diagnose breast cancer by embedding a diagnostic algorithm that examines the asymmetry of bilateral breast tissue. This retrospective study was approved by the institutional review board. A total of 115 patients who underwent breast surgery and had pathologically confirmed breast cancer were enrolled in this study. Two image pairs [230 pairs of bilateral breast digital breast tomosynthesis (DBT) images with 115 malignant tumors and contralateral tissue (M/N), and 115 bilateral normal areas (N/N)] were generated from each patient enrolled in this study. The proposed deep learning model is called bilateral asymmetrical detection (BilAD), which is a modified convolutional neural network (CNN) model of Xception with two-dimensional tensors for bilateral breast images. BilAD was trained to classify the differences between pairs of M/N and N/N datasets. The results of the BilAD model were compared to those of the unilateral control CNN model (uCNN). The results of BilAD and the uCNN were as follows: accuracy, 0.84 and 0.75; sensitivity, 0.73 and 0.58; and specificity, 0.93 and 0.92, respectively. The mean area under the receiver operating characteristic curve of BilAD was significantly higher than that of the uCNN ( p = 0.02): 0.90 and 0.84, respectively. The proposed deep learning model trained by embedding a diagnostic algorithm to examine the asymmetry of bilateral breast tissue improves the diagnostic accuracy for breast cancer.