Splitting expands the application range of Vision Transformer -- variable Vision Transformer (vViT)

Vision Transformer (ViT) has achieved outstanding results in computer vision. Although there are many Transformer-based architectures derived from the original ViT, the dimension of patches are often the same with each other. This disadvantage leads to a limited application range in the medical field because in the medical field, datasets whose dimension is different from each other; e.g. medical image, patients' personal information, laboratory test and so on. To overcome this limitation, we develop a new derived type of ViT termed variable Vision Transformer (vViT). The aim of this study is to introduce vViT and to apply vViT to radiomics using T1 weighted magnetic resonance image (MRI) of glioma. In the prediction of 365 days of survival among glioma patients using radiomics,vViT achieved 0.83, 0.82, 0.81, and 0.76 in sensitivity, specificity, accuracy, and AUC-ROC, respectively. vViT has the potential to handle different types of medical information at once.

[1]  Syed Talha Bukhari,et al.  Overall Survival Prediction of Glioma Patients With Multiregional Radiomics , 2022, Frontiers in Neuroscience.

[2]  Kwo-Sen Kuo,et al.  Assessing Deep Neural Networks as Probability Estimators , 2021, 2021 IEEE International Conference on Big Data (Big Data).

[3]  J. Mongan,et al.  The University of California San Francisco Preoperative Diffuse Glioma MRI Dataset. , 2021, Radiology. Artificial intelligence.

[4]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[5]  Jitender Saini,et al.  A Review of Radiomics and Deep Predictive Modeling in Glioma Characterization. , 2020, Academic radiology.

[6]  Uri Shaham,et al.  DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network , 2016, BMC Medical Research Methodology.

[7]  Andriy Fedorov,et al.  Computational Radiomics System to Decode the Radiographic Phenotype. , 2017, Cancer research.

[8]  P. Lambin,et al.  Radiomics: the bridge between medical imaging and personalized medicine , 2017, Nature Reviews Clinical Oncology.

[9]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[12]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[15]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.