Detection and Labeling of Vertebrae in MR Images Using Deep Learning with Clinical Annotations as Training Data

The purpose of this study was to investigate the potential of using clinically provided spine label annotations stored in a single institution image archive as training data for deep learning-based vertebral detection and labeling pipelines. Lumbar and cervical magnetic resonance imaging cases with annotated spine labels were identified and exported from an image archive. Two separate pipelines were configured and trained for lumbar and cervical cases respectively, using the same setup with convolutional neural networks for detection and parts-based graphical models to label the vertebrae. The detection sensitivity, precision and accuracy rates ranged between 99.1–99.8, 99.6–100, and 98.8–99.8% respectively, the average localization error ranges were 1.18–1.24 and 2.38–2.60 mm for cervical and lumbar cases respectively, and with a labeling accuracy of 96.0–97.0%. Failed labeling results typically involved failed S1 detections or missed vertebrae that were not fully visible on the image. These results show that clinically annotated image data from one image archive is sufficient to train a deep learning-based pipeline for accurate detection and labeling of MR images depicting the spine. Further, these results support using deep learning to assist radiologists in their work by providing highly accurate labels that only require rapid confirmation.

[1]  Yiqiang Zhan,et al.  Cross-Modality Vertebrae Localization and Labeling Using Learning-Based Approaches , 2015 .

[2]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[3]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[4]  Shu Liao,et al.  Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition , 2016, IEEE Transactions on Medical Imaging.

[5]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[6]  Nassir Navab,et al.  Online tracking of interventional devices for endovascular aortic repair , 2015, International Journal of Computer Assisted Radiology and Surgery.

[7]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[8]  Shuo Li,et al.  Multi-Modality Vertebra Recognition in Arbitrary Views Using 3D Deformable Hierarchical Model , 2015, IEEE Transactions on Medical Imaging.

[9]  Dumitru Erhan,et al.  Deep Neural Networks for Object Detection , 2013, NIPS.

[10]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Purang Abolmaesumi,et al.  Fast Automatic Vertebrae Detection and Localization in Pathological CT Scans - A Deep Learning Approach , 2015, MICCAI.

[12]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.

[13]  Shang-Hong Lai,et al.  Learning-Based Vertebra Detection and Iterative Normalized-Cut Segmentation for Spinal MRI , 2009, IEEE Transactions on Medical Imaging.

[14]  Ben Glocker,et al.  Vertebrae Localization in Pathological Spine CT via Dense Classification from Sparse Annotations , 2013, MICCAI.

[15]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[16]  Ronald M. Summers,et al.  Interleaved text/image Deep Mining on a large-scale radiology database , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Ayse Betül Oktay,et al.  Simultaneous Localization of Lumbar Vertebrae and Intervertebral Discs With SVM-Based MRF , 2013, IEEE Transactions on Biomedical Engineering.

[18]  Andrew Zisserman,et al.  Vertebrae Detection and Labelling in Lumbar MR Images , 2014 .

[19]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[20]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[21]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[22]  Cristian Lorenz,et al.  Automated model-based vertebra detection, identification, and segmentation in CT images , 2009, Medical Image Anal..

[23]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[24]  Florian Schulze,et al.  Automated landmarking and labeling of fully and partially scanned spinal columns in CT images , 2013, Medical Image Anal..

[25]  Hao Chen,et al.  Automatic Localization and Identification of Vertebrae in Spine CT via a Joint Learning Model with Deep Neural Networks , 2015, MICCAI.

[26]  Shuo Li,et al.  Multi-modal vertebrae recognition using Transformed Deep Convolution Network , 2016, Comput. Medical Imaging Graph..

[27]  Ben Glocker,et al.  Automatic Localization and Identification of Vertebrae in Arbitrary Field-of-View CT Scans , 2012, MICCAI.