Automatic Ischemic Stroke Lesion Segmentation from Computed Tomography Perfusion Images by Image Synthesis and Attention-Based Deep Neural Networks

Ischemic stroke lesion segmentation from Computed Tomography Perfusion (CTP) images is important for accurate diagnosis of stroke in acute care units. However, it is challenged by low image contrast and resolution of the perfusion parameter maps, in addition to the complex appearance of the lesion. To deal with this problem, we propose a novel framework based on synthesized pseudo Diffusion-Weighted Imaging (DWI) from perfusion parameter maps to obtain better image quality for more accurate segmentation. Our framework consists of three components based on Convolutional Neural Networks (CNNs) and is trained end-to-end. First, a feature extractor is used to obtain both a low-level and high-level compact representation of the raw spatiotemporal Computed Tomography Angiography (CTA) images. Second, a pseudo DWI generator takes as input the concatenation of CTP perfusion parameter maps and our extracted features to obtain the synthesized pseudo DWI. To achieve better synthesis quality, we propose a hybrid loss function that pays more attention to lesion regions and encourages high-level contextual consistency. Finally, we segment the lesion region from the synthesized pseudo DWI, where the segmentation network is based on switchable normalization and channel calibration for better performance. Experimental results showed that our framework achieved the top performance on ISLES 2018 challenge and: (1) our method using synthesized pseudo DWI outperformed methods segmenting the lesion from perfusion parameter maps directly; (2) the feature extractor exploiting additional spatiotemporal CTA images led to better synthesized pseudo DWI quality and higher segmentation accuracy; and (3) the proposed loss functions and network structure improved the pseudo DWI synthesis and lesion segmentation performance. The proposed framework has a potential for improving diagnosis and treatment of the ischemic stroke where access to real DWI scanning is limited.

[1]  Konstantinos Kamnitsas,et al.  Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation , 2017, IEEE Transactions on Medical Imaging.

[2]  Nancy J Fischbein,et al.  Comparison of Arterial Spin Labeling and Bolus Perfusion-Weighted Imaging for Detecting Mismatch in Acute Stroke , 2012, Stroke.

[3]  Olivier Salvado,et al.  Lesion segmentation from multimodal MRI using random forest following ischemic stroke , 2014, NeuroImage.

[4]  Christian S. Perone,et al.  Unsupervised domain adaptation for medical imaging segmentation with self-ensembling , 2018, NeuroImage.

[5]  Jonathan Rubin,et al.  Ischemic Stroke Lesion Segmentation in CT Perfusion Scans using Pyramid Pooling and Focal Loss , 2018, BrainLes@MICCAI.

[6]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[7]  Sotirios A. Tsaftaris,et al.  Multimodal MR Synthesis via Modality-Invariant Latent Representation , 2018, IEEE Transactions on Medical Imaging.

[8]  Ian B. Ross,et al.  Acute Ischemic Stroke: Imaging and Intervention , 2011 .

[9]  et al.,et al.  ISLES 2015 ‐ A public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI , 2017, Medical Image Anal..

[10]  Jose Dolz,et al.  Boundary loss for highly unbalanced segmentation , 2018, MIDL.

[11]  Tao Song,et al.  Integrated Extractor, Generator and Segmentor for Ischemic Stroke Lesion Segmentation , 2018, BrainLes@MICCAI.

[12]  Shaohua Kevin Zhou,et al.  Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network , 2015, MICCAI.

[13]  Zhi-Qin John Xu,et al.  Training behavior of deep neural network in frequency domain , 2018, ICONIP.

[14]  Ping Luo,et al.  Differentiable Learning-to-Normalize via Switchable Normalization , 2018, ICLR.

[15]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[16]  Dazhe Zhao,et al.  Segmentation of Ischemic Stroke Lesions in Multi-spectral MR Images Using Weighting Suppressed FCM and Three Phase Level Set , 2015, Brainles@MICCAI.

[17]  D. Mezzapesa,et al.  Multimodal MR examination in acute ischemic stroke , 2006, Neuroradiology.

[18]  Eric Granger,et al.  Constrained‐CNN losses for weakly supervised segmentation☆ , 2018, Medical Image Anal..

[19]  Klaus H. Maier-Hein,et al.  No New-Net , 2018, 1809.10483.

[20]  Yiming Li,et al.  Semi-Supervised Brain Lesion Segmentation with an Adapted Mean Teacher Model , 2019, IPMI.

[21]  Alejandro F. Frangi,et al.  Simulation and Synthesis in Medical Imaging , 2018, IEEE Trans. Medical Imaging.

[22]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[23]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[24]  Andriy Myronenko,et al.  3D MRI brain tumor segmentation using autoencoder regularization , 2018, BrainLes@MICCAI.

[25]  Aritra Ghosh,et al.  Robust Loss Functions under Label Noise for Deep Neural Networks , 2017, AAAI.

[26]  Dinggang Shen,et al.  Reconstruction of 7T-Like Images From 3T MRI , 2016, IEEE Transactions on Medical Imaging.

[27]  Snehashis Roy,et al.  MR contrast synthesis for lesion segmentation , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[28]  Hayit Greenspan,et al.  GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification , 2018, Neurocomputing.

[29]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Abdelhafid Bessaid,et al.  Segmentation of ischemic stroke area from CT brain images , 2016, 2016 International Symposium on Signal, Image, Video and Communications (ISIVC).

[32]  K. Katada,et al.  Preliminary study of time maximum intensity projection computed tomography imaging for the detection of early ischemic change in patient with acute ischemic stroke , 2018, Medicine.

[33]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  C. Garbay,et al.  Multimodal MRI segmentation of ischemic stroke lesions , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[35]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[36]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[37]  Jose Dolz,et al.  Dense Multi-path U-Net for Ischemic Stroke Lesion Segmentation in Multiple Image Modalities , 2018, BrainLes@MICCAI.

[38]  Glyn W. Humphreys,et al.  NeuroImage: Clinical Automated delineation of stroke lesions using brain CT images , 2022 .

[39]  Ninon Burgos,et al.  Attenuation Correction Synthesis for Hybrid PET-MR Scanners: Application to Brain Studies , 2014, IEEE Transactions on Medical Imaging.

[40]  Vijayan K. Asari,et al.  Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation , 2018, ArXiv.

[41]  Pengbo Liu,et al.  Stroke Lesion Segmentation with 2D Novel CNN Pipeline and Novel Loss Function , 2018, BrainLes@MICCAI.

[42]  Snehashis Roy,et al.  Random forest regression for magnetic resonance image synthesis , 2017, Medical Image Anal..

[43]  Nils Daniel Forkert,et al.  Classifiers for Ischemic Stroke Lesion Segmentation: A Comparison Study , 2015, PloS one.

[44]  S. Allassonnière,et al.  Medical image analysis methods in MR/CT-imaged acute-subacute ischemic stroke lesion: Segmentation, prediction and insights into dynamic evolution simulation models. A critical appraisal , 2012, NeuroImage: Clinical.

[45]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[46]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[47]  Heinz Handels,et al.  Ischemic stroke lesion segmentation in multi-spectral MR images with support vector machine classifiers , 2014, Medical Imaging.

[48]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[49]  Letícia Rittner,et al.  V-Net and U-Net for Ischemic Stroke Lesion Segmentation in a Small Dataset of Perfusion Data , 2018, BrainLes@MICCAI.

[50]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[51]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Zhiming Luo,et al.  Weighted Res-UNet for High-Quality Retina Vessel Segmentation , 2018, 2018 9th International Conference on Information Technology in Medicine and Education (ITME).

[53]  Andrew L Beers,et al.  ISLES 2016 and 2017-Benchmarking Ischemic Stroke Lesion Outcome Prediction Based on Multispectral MRI , 2018, Front. Neurol..

[54]  M. Wintermark,et al.  Perfusion CT and acute stroke imaging: foundations, applications, and literature review. , 2015, Journal of neuroradiology. Journal de neuroradiologie.

[55]  Ganapathy Krishnamurthi,et al.  Fully Automatic Segmentation for Ischemic Stroke Using CT Perfusion Maps , 2018, BrainLes@MICCAI.

[56]  Dinggang Shen,et al.  Medical Image Synthesis with Deep Convolutional Adversarial Networks , 2018, IEEE Transactions on Biomedical Engineering.

[57]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.