Detection of high-grade atypia nuclei in breast cancer imaging

Along with mitotic count, nuclear pleomorphism or nuclear atypia is an important criterion for the grading of breast cancer in histopathology. Though some works have been done in mitosis detection (ICPR 2012,1 MICCAI 2013,2 and ICPR 2014), not much work has been dedicated to automated nuclear atypia grading, especially the most difficult task of detection of grade 3 nuclei. We propose the use of Convolutional Neural Networks for the automated detection of cell nuclei, using images from the three grades of breast cancer for training. The images were obtained from ICPR contests. Additional manual annotation was performed to classify pixels into five classes: stroma, nuclei, lymphocytes, mitosis and fat. At total of 3,000 thumbnail images of 101 × 101 pixels were used for training. By dividing this training set in an 80/20 ratio we could obtain good training results (around 90%). We tested our CNN on images of the three grades which were not in the training set. High grades nuclei were correctly classified. We then thresholded the classification map and performed basic analysis to keep only rounded objects. Our results show that mostly all atypical nuclei were correctly detected.