Vertebrae Identification and Localization Utilizing Fully Convolutional Networks and a Hidden Markov Model

Automated identification and localization of vertebrae in spinal computed tomography (CT) imaging is a complicated hybrid task. This task requires detecting and indexing a long sequence in a 3-D image, and both image feature extraction and sequence modeling are needed to address the problem. In this paper, the powerful fully convolutional neural network (FCN) technique performs both of these tasks simultaneously because FCNs directly encode and decode the spatial interdependence of different components in images. The key module of our proposed framework is a 3-D FCN trained in an end-to-end manner at the spine level to capture the long-range contextual information in CT volumes. The large increase in the calculation due to the full-size image inputs is alleviated by the scale-down of the inputs and the use of an auxiliary FCN to compensate for the loss of details. The composite network pipeline design enables the integration of local image details and global image patterns. Furthermore, explicit spatial and sequential constraints are imposed by the hidden Markov model (HMM) for a higher robustness and a clearer interpretation of network outputs. The proposed framework is quantitatively evaluated on the public dataset from the MICCAI 2014 Computational Challenge on Vertebrae Localization and Identification and demonstrates an identification rate (within 20 mm) of 94.67%, a mean identification rate of 87.97%, and a mean error distance of 2.56 mm on the test set, thus achieving the highest performance reported on this dataset.

[1]  Seungkyu Lee,et al.  Vertebrae localization in CT using both local and global symmetry features , 2017, Comput. Medical Imaging Graph..

[2]  Ronald M. Summers,et al.  Detection of Vertebral Body Fractures Based on Cortical Shell Unwrapping , 2012, MICCAI.

[3]  Ben Glocker,et al.  Vertebrae Localization in Pathological Spine CT via Dense Classification from Sparse Annotations , 2013, MICCAI.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[6]  Mohammed Benjelloun,et al.  Fast 3D Spine Reconstruction of Postoperative Patients Using a Multilevel Statistical Model , 2012, MICCAI.

[7]  Qian Wang,et al.  Deep embedding convolutional neural network for synthesizing CT image from T1‐Weighted MR image , 2018, Medical Image Anal..

[8]  Hao Chen,et al.  Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection , 2017, IEEE Transactions on Biomedical Engineering.

[9]  Shaoting Zhang,et al.  Recognizing End-Diastole and End-Systole Frames via Deep Temporal Regression Network , 2016, MICCAI.

[10]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[11]  Hao Chen,et al.  Automatic Fetal Ultrasound Standard Plane Detection Using Knowledge Transferred Recurrent Neural Networks , 2015, MICCAI.

[12]  Jiebo Luo,et al.  Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information , 2018, IEEE Transactions on Medical Imaging.

[13]  Kumaradevan Punithakumar,et al.  Vertebral Body Segmentation in MRI via Convex Relaxation and Distribution Matching , 2012, MICCAI.

[14]  Ben Glocker,et al.  Automatic Localization and Identification of Vertebrae in Arbitrary Field-of-View CT Scans , 2012, MICCAI.

[15]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[16]  Le Lu,et al.  Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function , 2017, ArXiv.

[17]  Sebastian P. M. Dries,et al.  Spine Detection and Labeling Using a Parts-Based Graphical Model , 2007, IPMI.

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Benoit M. Dawant,et al.  Automatic Lumbar Vertebral Identification Using Surface-Based Registration , 2001, J. Biomed. Informatics.

[21]  Yuichiro Hayashi,et al.  An application of cascaded 3D fully convolutional networks for medical image segmentation , 2018, Comput. Medical Imaging Graph..

[22]  Joseph E. Burns.  Imaging of the Spine: A Medical and Physical Perspective , 2015 .

[23]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[24]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[25]  Yiqiang Zhan,et al.  Robust MR Spine Detection Using Hierarchical Learning and Local Articulated Model , 2012, MICCAI.

[26]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[27]  Ronald M. Summers,et al.  A multi-center milestone study of clinical vertebral CT segmentation , 2016, Comput. Medical Imaging Graph..

[28]  Purang Abolmaesumi,et al.  Fast Automatic Vertebrae Detection and Localization in Pathological CT Scans - A Deep Learning Approach , 2015, MICCAI.

[29]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Hao Chen,et al.  Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss , 2018, IJCAI.

[31]  Daguang Xu,et al.  Automatic Vertebra Labeling in Large-Scale 3D CT using Deep Image-to-Image Network with Message Passing and Sparsity Regularization , 2017, IPMI.

[32]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[33]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[34]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[35]  Jun Ma,et al.  Hierarchical segmentation and identification of thoracic vertebra using learning-based edge detection and coarse-to-fine deformable model , 2010, Comput. Vis. Image Underst..

[36]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[37]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[38]  Hao Chen,et al.  3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes , 2016, MICCAI.

[39]  Hao Chen,et al.  Automatic Localization and Identification of Vertebrae in Spine CT via a Joint Learning Model with Deep Neural Networks , 2015, MICCAI.