Classification of Brain Tumours in MR Images using Deep Spatiospatial Models

A brain tumour is a mass or cluster of abnormal cells in the brain, which has the possibility of becoming life-threatening because of its ability to invade neighbouring tissues and also form metastases. An accurate diagnosis is essential for successful treatment planning and magnetic resonance imaging is the principal imaging modality for diagnostic of brain tumours and their extent. Deep Learning methods in computer vision applications have shown significant improvement in recent years, most of which can be credited to the fact that a sizeable amount of data is available to train models on, and the improvements in the model architectures yielding better approximations in a supervised setting. Classifying tumours using such deep learning methods has made significant progress with the availability of open datasets with reliable annotations. Typically those methods are either 3D models, which use 3D volumetric MRIs or even 2D models considering each slice separately. However, by treating one spatial dimension separately or by considering the slices as a sequence of images over time, spatiotemporal models can be employed as "spatiospatial" models for this task. These models have the capabilities of learning specific spatial and temporal relationship, while reducing computational costs. This paper uses two spatiotemporal models, ResNet (2+1)D and ResNet Mixed Convolution, to classify different types of brain tumours. It was observed that both these models performed superior to the pure 3D convolutional model, ResNet18. Furthermore, it was also observed that pre-training the models on a different, even unrelated dataset before training them for the task of tumour classification improves the performance. Finally, Pre-trained ResNet Mixed Convolution was observed to be the best model in these experiments, achieving a macro F1-score of 0.9345 and a test accuracy of 96.98%, while at the same time being the model with the least computational cost.

[1]  Taghi M. Khoshgoftaar,et al.  Survey on deep learning with class imbalance , 2019, J. Big Data.

[2]  Khan M. Iftekharuddin,et al.  Brain Tumor Classification Using 3D Convolutional Neural Network , 2019, BrainLes@MICCAI.

[3]  P. Trott,et al.  International Classification of Diseases for Oncology , 1977 .

[4]  Hui Xiong,et al.  A Comprehensive Survey on Transfer Learning , 2019, Proceedings of the IEEE.

[5]  C. Muir,et al.  International Classification of Diseases for Oncology , 1990 .

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Luc Taillandier,et al.  Quantitative morphological magnetic resonance imaging follow-up of low-grade glioma: a plea for systematic measurement of growth rates. , 2012, Neurosurgery.

[8]  R. Jenkins,et al.  Genetics of adult glioma. , 2012, Cancer genetics.

[9]  Jie Yang,et al.  Deep Learning and Multi-Sensor Fusion for Glioma Classification Using Multistream 2D Convolutional Networks , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[10]  M. Berger,et al.  Survival and low-grade glioma: the emergence of genetic information. , 2015, Neurosurgical focus.

[11]  Hao Wu,et al.  Mixed Precision Training , 2017, ICLR.

[12]  Yann LeCun,et al.  A Closer Look at Spatiotemporal Convolutions for Action Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Emilio Soria Olivas,et al.  Handbook of Research on Machine Learning Applications and Trends : Algorithms , Methods , and Techniques , 2009 .

[14]  Ying Zhuge,et al.  Automated Glioma Grading on Conventional MRI images Using Deep Convolutional Neural Networks. , 2020, Medical physics.

[15]  Raymond Sawaya,et al.  Necrosis and Glioblastoma: A Friend or a Foe? A Review and a Hypothesis , 2002, Neurosurgery.

[16]  Sébastien Ourselin,et al.  TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning , 2020, Comput. Methods Programs Biomed..

[17]  C. Nimsky,et al.  Cellular characterization of the peritumoral edema zone in malignant brain tumors , 2009, Cancer science.

[18]  Klaus H. Maier-Hein,et al.  nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation , 2018, Bildverarbeitung für die Medizin.

[19]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[20]  Yang Yang,et al.  Glioma Grading on Conventional MR Images: A Deep Learning Study With Transfer Learning , 2018, Front. Neurosci..

[21]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[22]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[23]  Andreas Nürnberger,et al.  Fine-tuning deep learning model parameters for improved super-resolution of dynamic MRI with prior-knowledge , 2021, Artificial intelligence in medicine.

[24]  A. Ng,et al.  Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists , 2018, PLoS medicine.

[25]  Ezzeddine Zagrouba,et al.  Glioma classification via MR images radiomics analysis , 2021, The Visual Computer.

[26]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[27]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[28]  Gavriel Salomon,et al.  T RANSFER OF LEARNING , 1992 .

[29]  Tong Boon Tang,et al.  CNN-LSTM: Cascaded Framework For Brain Tumour Classification , 2018, 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES).

[30]  Ali Wali,et al.  Deep Multi-Scale 3D Convolutional Neural Network (CNN) for MRI Gliomas Brain Tumor Classification , 2020, Journal of Digital Imaging.