Localizing 2D Ultrasound Probe from Ultrasound Image Sequences Using Deep Learning for Volume Reconstruction

This paper presents an ultrasound (US) volume reconstruction method only from US image sequences using deep learning. The proposed method employs the convolutional neural network (CNN) to estimate the position of a 2D US probe only from US images. Our CNN model consists of two networks: feature extraction and motion estimation. We also introduce the consistency loss function to enforce. Through a set of experiments using US image sequence datasets with ground-truth motion measured by a motion capture system, we demonstrate that the proposed method exhibits the efficient performance on probe localization and volume reconstruction compared with the conventional method.

[1]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[2]  P.C. Pedersen,et al.  An inertial-optical tracking system for portable, quantitative, 3D ultrasound , 2008, 2008 IEEE Ultrasonics Symposium.

[3]  Hedyeh Rafii-Tari,et al.  Panorama Ultrasound for Guiding Epidural Anesthesia: A Feasibility Study , 2011, IPCAI.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Alfredo Illanes,et al.  Novel Similarity Metric for Image-Based Out-Of-Plane Motion Estimation in 3D Ultrasound , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[6]  Bo Wang,et al.  Towards an Ultrasound Probe with Vision: Structured Light to Determine Surface Orientation , 2011, AE-CAI.

[7]  Oisin Mac Aodha,et al.  Unsupervised Monocular Depth Estimation with Left-Right Consistency , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Thomas Lange,et al.  Automatic Calibration of 3D Ultrasound Probes , 2011, Bildverarbeitung für die Medizin.

[9]  Mehrdad Salehi,et al.  Deep Learning for Sensorless 3D Freehand Ultrasound Imaging , 2017, MICCAI.

[10]  H. Meinzer,et al.  Effect of 3D ultrasound probes on the accuracy of electromagnetic tracking systems. , 2006, Ultrasound in medicine & biology.

[11]  Hyun-Jae Kang,et al.  Multi-DoF probe trajectory reconstruction with local sensors for 2D-to-3D ultrasound , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[12]  Satoshi Kondo,et al.  A Probe-Camera System for 3D Ultrasound Image Reconstruction , 2017, BIVPCS/POCUS@MICCAI.

[13]  Pierre Hellier,et al.  A fully automatic calibration procedure for freehand 3D ultrasound , 2002, Proceedings IEEE International Symposium on Biomedical Imaging.

[14]  Takafumi Aoki,et al.  Probe localization using structure from motion for 3D ultrasound image reconstruction , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[15]  Navneet Kumar,et al.  3D freehand ultrasound without external tracking using deep learning , 2018, Medical Image Anal..

[16]  Graham M. Treece,et al.  Engineering a freehand 3D ultrasound system , 2003, Pattern Recognit. Lett..

[17]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Matthew W. Gilbertson,et al.  Probe Localization for Freehand 3D Ultrasound by Tracking Skin Features , 2014, MICCAI.

[19]  T. Nelson,et al.  Three-dimensional ultrasound imaging. , 1998, Ultrasound in medicine & biology.