Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information

Automatic vertebrae identification and localization from arbitrary computed tomography (CT) images is challenging. Vertebrae usually share similar morphological appearance. Because of pathology and the arbitrary field-of-view of CT scans, one can hardly rely on the existence of some anchor vertebrae or parametric methods to model the appearance and shape. To solve the problem, we argue that: 1) one should make use of the short-range contextual information, such as the presence of some nearby organs (if any), to roughly estimate the target vertebrae; and 2) due to the unique anatomic structure of the spine column, vertebrae have fixed sequential order, which provides the important long-range contextual information to further calibrate the results. We propose a robust and efficient vertebrae identification and localization system that can inherently learn to incorporate both the short- and long-range contextual information in a supervised manner. To this end, we develop a multi-task 3-D fully convolutional neural network to effectively extract the short-range contextual information around the target vertebrae. For the long-range contextual information, we propose a multi-task bidirectional recurrent neural network to encode the spatial and contextual information among the vertebrae of the visible spine column. We demonstrate the effectiveness of the proposed approach on a challenging data set, and the experimental results show that our approach outperforms the state-of-the-art methods by a significant margin.

[1]  Ronald M. Summers,et al.  Detection of Vertebral Body Fractures Based on Cortical Shell Unwrapping , 2012, MICCAI.

[2]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[3]  Ben Glocker,et al.  Vertebrae Localization in Pathological Spine CT via Dense Classification from Sparse Annotations , 2013, MICCAI.

[4]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[5]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[6]  Oscar E. Ruiz,et al.  Robust CT to US 3D-3D Registration by Using Principal Component Analysis and Kalman Filtering , 2015, CSI@MICCAI.

[7]  Mohammed Benjelloun,et al.  Fast 3D Spine Reconstruction of Postoperative Patients Using a Multilevel Statistical Model , 2012, MICCAI.

[8]  Vipin Chaudhary,et al.  Vertebral Column Localization, Labeling, and Segmentation , 2015 .

[9]  Le Lu,et al.  Pancreas Segmentation in MRI Using Graph-Based Decision Fusion on Convolutional Neural Networks , 2016, MICCAI.

[10]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[11]  Jun Ma,et al.  Hierarchical segmentation and identification of thoracic vertebra using learning-based edge detection and coarse-to-fine deformable model , 2010, Comput. Vis. Image Underst..

[12]  Bostjan Likar,et al.  Manual and Computer-Assisted Pedicle Screw Placement Plans: A Quantitative Comparison , 2016, CSI@MICCAI.

[13]  Shaoting Zhang,et al.  Recognizing End-Diastole and End-Systole Frames via Deep Temporal Regression Network , 2016, MICCAI.

[14]  Vipin Chaudhary,et al.  Compression fracture diagnosis in lumbar: a clinical CAD system , 2013, International Journal of Computer Assisted Radiology and Surgery.

[15]  Joseph E. Burns.  Imaging of the Spine: A Medical and Physical Perspective , 2015 .

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[18]  Ben Glocker,et al.  Automatic Localization and Identification of Vertebrae in Arbitrary Field-of-View CT Scans , 2012, MICCAI.

[19]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[20]  Yongxin Yang,et al.  Deep Multi-task Representation Learning: A Tensor Factorisation Approach , 2016, ICLR.

[21]  Hao Chen,et al.  Automatic Fetal Ultrasound Standard Plane Detection Using Knowledge Transferred Recurrent Neural Networks , 2015, MICCAI.

[22]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Ronald M. Summers,et al.  Detection of Degenerative Osteophytes of the Spine on PET/CT Using Region-Based Convolutional Neural Networks , 2016, CSI@MICCAI.

[24]  R. Kumar,et al.  Robotic Assistance and Intervention in Spine Surgery , 2015 .

[25]  Le Lu,et al.  Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function , 2017, ArXiv.

[26]  Hao Chen,et al.  Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[27]  Sebastian P. M. Dries,et al.  Spine Detection and Labeling Using a Parts-Based Graphical Model , 2007, IPMI.

[28]  Loren Cobb,et al.  A Fast, Accurate, and Reliable Reconstruction Method of the Lumbar Spine Vertebrae Using Positional MRI , 2013, Annals of Biomedical Engineering.

[29]  Daguang Xu,et al.  Automatic Vertebra Labeling in Large-Scale 3D CT using Deep Image-to-Image Network with Message Passing and Sparsity Regularization , 2017, IPMI.

[30]  Purang Abolmaesumi,et al.  Fast Automatic Vertebrae Detection and Localization in Pathological CT Scans - A Deep Learning Approach , 2015, MICCAI.

[31]  Yiqiang Zhan,et al.  Robust MR Spine Detection Using Hierarchical Learning and Local Articulated Model , 2012, MICCAI.

[32]  Lin Yang,et al.  Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation , 2016, NIPS.

[33]  Ronald M. Summers,et al.  Progressive and Multi-path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images , 2017, MICCAI.

[34]  Richard A. Robb,et al.  Toward Virtual Modeling and Templating for Enhanced Spine Surgery Planning , 2015 .

[35]  Kumaradevan Punithakumar,et al.  Vertebral Body Segmentation in MRI via Convex Relaxation and Distribution Matching , 2012, MICCAI.

[36]  Dorin Comaniciu,et al.  Detection of 3D Spinal Geometry Using Iterated Marginal Space Learning , 2010, MCV.

[37]  S. Schafer,et al.  Automatic localization of vertebral levels in x-ray fluoroscopy using 3D-2D registration: a tool to reduce wrong-site surgery , 2012, Physics in medicine and biology.

[38]  Hao Chen,et al.  Automatic Localization and Identification of Vertebrae in Spine CT via a Joint Learning Model with Deep Neural Networks , 2015, MICCAI.