A Novel Framework for 3D-2D Vertebra Matching

3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.

[1]  Zheng Zhang,et al.  MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems , 2015, ArXiv.

[2]  Bostjan Likar,et al.  A review of 3D/2D registration methods for image-guided interventions , 2012, Medical Image Anal..

[3]  Xianming Liu,et al.  Learning Temporal Dynamics for Video Super-Resolution: A Deep Learning Approach , 2018, IEEE Transactions on Image Processing.

[4]  Xianming Liu,et al.  Greedy Batch-Based Minimum-Cost Flows for Tracking Multiple Objects , 2017, IEEE Transactions on Image Processing.

[5]  Jiri Matas,et al.  Learning Discriminative Affine Regions via Discriminability , 2017, ArXiv.

[6]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Jian Dong,et al.  Attentive Contexts for Object Detection , 2016, IEEE Transactions on Multimedia.

[8]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[9]  Mohammed Benjelloun,et al.  Vertebra identification using template matching modelmp and $$K$$K-means clustering , 2014, International Journal of Computer Assisted Radiology and Surgery.

[10]  Dacheng Tao,et al.  Geometry-Aware Scene Text Detection with Instance Transformation Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Jinjun Xiong,et al.  Revisiting RCNN: On Awakening the Classification Power of Faster RCNN , 2018, ECCV.

[12]  Zhengrong Liang,et al.  A fast ray-tracing technique for TCT and ECT studies , 1999, 1999 IEEE Nuclear Science Symposium. Conference Record. 1999 Nuclear Science Symposium and Medical Imaging Conference (Cat. No.99CH37019).

[13]  R. Kikinis,et al.  Image-guided surgery. , 1999, Scientific American.

[14]  Dacheng Tao,et al.  Subspaces Indexing Model on Grassmann Manifold for Image Search , 2011, IEEE Transactions on Image Processing.

[15]  Jun Yu,et al.  FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification , 2018, ECCV.

[16]  Mohammed Benjelloun,et al.  Fully automatic vertebra detection in x-ray images based on multi-class SVM , 2012, Medical Imaging.

[17]  Yunchao Wei,et al.  Perceptual Generative Adversarial Networks for Small Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Wei Shen,et al.  Automatic localization of vertebrae based on convolutional neural networks , 2015, Medical Imaging.

[19]  Wen Gao,et al.  Interacting Tracklets for Multi-Object Tracking , 2018, IEEE Transactions on Image Processing.

[20]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[21]  Jinjun Xiong,et al.  TS2C: Tight Box Mining with Surrounding Segmentation Context for Weakly Supervised Object Detection , 2018, ECCV.

[22]  R. Siddon.  Fast calculation of the exact radiological path for a three-dimensional CT array. , 1985, Medical physics.

[23]  Pascal Fua,et al.  Globally Consistent Multi-People Tracking using Motion Patterns , 2016, ArXiv.

[24]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[25]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[26]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Pascal Fua,et al.  Tracking Interacting Objects Using Intertwined Flows , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Bjorn De Sutter,et al.  A Fast Algorithm to Calculate the Exact Radiological Path through a Pixel or Voxel Space , 1998 .

[29]  Nagarajan Kandasamy,et al.  Plastimatch—An Open-Source Software for Radiotherapy Imaging , 2014 .

[30]  Jinjun Xiong,et al.  Decoupled Classification Refinement: Hard False Positive Suppression for Object Detection , 2018, ArXiv.

[31]  Huimin Ma,et al.  3D Object Proposals for Accurate Object Class Detection , 2015, NIPS.

[32]  Hamed Sari-Sarraf,et al.  Customized Hough transform for robust segmentation of cervical vertebrae from X-ray images , 2002, Proceedings Fifth IEEE Southwest Symposium on Image Analysis and Interpretation.

[33]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..