Automatic X-ray landmark detection and shape segmentation via data-driven joint estimation of image displacements

In this paper, we propose a new method for fully-automatic landmark detection and shape segmentation in X-ray images. To detect landmarks, we estimate the displacements from some randomly sampled image patches to the (unknown) landmark positions, and then we integrate these predictions via a voting scheme. Our key contribution is a new algorithm for estimating these displacements. Different from other methods where each image patch independently predicts its displacement, we jointly estimate the displacements from all patches together in a data driven way, by considering not only the training data but also geometric constraints on the test image. The displacements estimation is formulated as a convex optimization problem that can be solved efficiently. Finally, we use the sparse shape composition model as the a priori information to regularize the landmark positions and thus generate the segmented shape contour. We validate our method on X-ray image datasets of three different anatomical structures: complete femur, proximal femur and pelvis. Experiments show that our method is accurate and robust in landmark detection, and, combined with the shape model, gives a better or comparable performance in shape segmentation compared to state-of-the art methods. Finally, a preliminary study using CT data shows the extensibility of our method to 3D data.

[1]  Weiliang Xu,et al.  Segmentation of radiographic images under topological constraints: application to the femur , 2010, International Journal of Computer Assisted Radiology and Surgery.

[2]  Rhodri H. Davies,et al.  Learning Shape: Optimal Models for Analysing Natural Variability , 2004 .

[3]  Dorin Comaniciu,et al.  Marginal Space Learning for Efficient Detection of 2D/3D Anatomical Structures in Medical Images , 2009, IPMI.

[4]  J. CandesE.,et al.  Near-Optimal Signal Recovery From Random Projections , 2006 .

[5]  Guoyan Zheng,et al.  Fully Automatic X-Ray Image Segmentation via Joint Estimation of Image Displacements , 2013, MICCAI.

[6]  Timothy F. Cootes,et al.  A mixture model for representing shape variation , 1999, Image Vis. Comput..

[7]  Guoyan Zheng Expectation Conditional Maximization-Based Deformable Shape Registration , 2013, CAIP.

[8]  Timothy F. Cootes,et al.  Accurate Fully Automatic Femur Segmentation in Pelvic Radiographs Using Regression Voting , 2012, MICCAI.

[9]  Yoshinobu Sato,et al.  Automated Segmentation of the Femur and Pelvis from 3D CT Data of Diseased Hip Using Hierarchical Statistical Shape Model of Joint Structure , 2009, MICCAI.

[10]  Volker Kuhn,et al.  Proximal femur segmentation in conventional pelvic x ray. , 2008, Medical physics.

[11]  Guoyan Zheng,et al.  Unsupervised Reconstruction of a Patient-Specific Surface Model of a Proximal Femur from Calibrated Fluoroscopic Images , 2007, MICCAI.

[12]  Paul Suetens,et al.  Active Shape Model-Based Segmentation of Digital X-ray Images , 1999, MICCAI.

[13]  Antonio Criminisi,et al.  Regression Forests for Efficient Anatomy Detection and Localization in CT Studies , 2010, MCV.

[14]  Dorin Comaniciu,et al.  Shape Regression Machine , 2007, IPMI.

[15]  Renaud Keriven,et al.  Shape Priors using Manifold Learning Techniques , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[16]  Guoyan Zheng,et al.  Automatic Extraction of Proximal Femur Contours from Calibrated X-Ray Images Using 3D Statistical Models , 2008, MIAR.

[17]  Dorin Comaniciu,et al.  Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features , 2008, IEEE Transactions on Medical Imaging.

[18]  Sebastian P. M. Dries,et al.  Spine Detection and Labeling Using a Parts-Based Graphical Model , 2007, IPMI.

[19]  Dimitris N. Metaxas,et al.  Deformable segmentation via sparse representation and dictionary learning , 2012, Medical Image Anal..

[20]  Guoyan Zheng,et al.  Automatic extraction of proximal femur contours from calibrated X‐ray images using 3D statistical models: an in vitro study , 2009, The international journal of medical robotics + computer assisted surgery : MRCAS.

[21]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[22]  Junzhou Huang,et al.  Sparse shape composition: A new framework for shape prior modeling , 2011, CVPR 2011.

[23]  Marleen de Bruijne,et al.  2D-3D shape reconstruction of the distal femur from stereo X-ray imaging using statistical shape models , 2011, Medical Image Anal..

[24]  Antonio Criminisi,et al.  Fast Multiple Organ Detection and Localization in Whole-Body MR Dixon Sequences , 2011, MICCAI.

[25]  Guoyan Zheng,et al.  A 2D/3D correspondence building method for reconstruction of a patient-specific 3D bone surface model using point distribution models and calibrated X-ray images , 2009, Medical Image Anal..

[26]  Rebecca Smith,et al.  A hierarchical method based on active shape models and directed Hough transform for segmentation of noisy biomedical images; application in segmentation of pelvic X-ray images , 2009, BMC Medical Informatics Decis. Mak..

[27]  Yi Yang,et al.  3D human pose recovery from image by efficient visual feature selection , 2011, Comput. Vis. Image Underst..

[28]  R Burgkart,et al.  Intraoperative, fluoroscopy‐based planning for complex osteotomies of the proximal femur , 2005, The international journal of medical robotics + computer assisted surgery : MRCAS.

[29]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Ying Chen,et al.  Automatic Extraction of Femur Contours from Hip X-Ray Images , 2005, CVBIA.

[31]  Juergen Gall,et al.  Class-specific Hough forests for object detection , 2009, CVPR.

[32]  C. Taylor,et al.  Active shape models - 'Smart Snakes'. , 1992 .

[33]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[34]  Horst Bischof,et al.  Generalized sparse MRF appearance models , 2010, Image Vis. Comput..

[35]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[36]  D. Donoho For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[37]  Tom Vercauteren,et al.  Diffeomorphic demons: Efficient non-parametric image registration , 2009, NeuroImage.

[38]  David Cristinacce,et al.  Automatic feature localisation with constrained local models , 2008, Pattern Recognit..

[39]  Christoph Schnörr,et al.  A Study of Parts-Based Object Class Detection Using Complete Graphs , 2010, International Journal of Computer Vision.

[40]  Xuelong Li,et al.  Estimating patient-specific shape prior for medical image segmentation , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[41]  Rasmus Larsen,et al.  Sparse Decomposition and Modeling of Anatomical Shape Variation , 2007, IEEE Transactions on Medical Imaging.

[42]  Yousef Saad,et al.  Trace optimization and eigenproblems in dimension reduction methods , 2011, Numer. Linear Algebra Appl..

[43]  Horst Bischof,et al.  Global localization of 3D anatomical structures by pre-filtered Hough Forests and discrete optimization , 2013, Medical Image Anal..

[44]  Timothy F. Cootes,et al.  Fully Automatic Segmentation of the Proximal Femur Using Random Forest Regression Voting , 2013, IEEE Transactions on Medical Imaging.

[45]  Martin Styner,et al.  Evaluation of 3D Correspondence Methods for Model Building , 2003, IPMI.