Deep Learning of Spatiotemporal Filtering for Fast Super-Resolution Ultrasound Imaging

Super-resolution ultrasound (SR-US) imaging is a new technique that breaks the diffraction limit and allows visualization of microvascular structures down to tens of micrometers. The image processing methods for the spatiotemporal filtering needed in SR-US, such as singular value decomposition (SVD), are computationally burdensome and performed offline. Deep learning has been applied to many biomedical imaging problems, and trained neural networks have been shown to process an image in milliseconds. The goal of this study was to evaluate the effectiveness of deep learning to realize a spatiotemporal filter in the context of SR-US processing. A 3-D convolutional neural network (3DCNN) was trained on <italic>in vitro</italic> and <italic>in vivo</italic> data sets using SVD as ground truth in tissue clutter reduction. <italic>In vitro</italic> data were obtained from a tissue-mimicking flow phantom, and <italic>in vivo</italic> data were collected from murine tumors of breast cancer. Three training techniques were studied: training with <italic>in vitro</italic> data sets, training with <italic>in vivo</italic> data sets, and transfer learning with initial training on <italic>in vitro</italic> data sets followed by fine-tuning with <italic>in vivo</italic> data sets. The neural network trained with <italic>in vitro</italic> data sets followed by fine-tuning with <italic>in vivo</italic> data sets had the highest accuracy at 88.0%. The SR-US images produced with deep learning allowed visualization of vessels as small as <inline-formula> <tex-math notation="LaTeX">$25~\mu \text{m}$ </tex-math></inline-formula> in diameter, which is below the diffraction limit (wavelength of <inline-formula> <tex-math notation="LaTeX">$110~\mu \text{m}$ </tex-math></inline-formula> at 14 MHz). The performance of the 3DCNN was encouraging for real-time SR-US imaging with an average processing frame rate for <italic>in vivo</italic> data of 51 Hz with GPU acceleration.