Computer-aided Detection and Visualization of Pulmonary Embolism Using a Novel, Compact, and Informative Image Representation

Diagnosing pulmonary embolism (PE) and excluding disorders that may clinically and radiologically simulate PE poses a challenging task for both human and machine perception. In this paper, we propose a novel vessel-oriented image representation (VOIR) that can improve the machine perception of PE through a consistent, compact, and discriminative image representation, and can also improve radiologists’ diagnostic capabilities for PE assessment by serving as the backbone of an effective PE visualization system. Specifically, our image representation can be used to train more effective convolutional neural networks for distinguishing PE from PE mimics, and also allows radiologists to inspect the vessel lumen from multiple perspectives, so that they can report filling defects (PE), if any, with confidence. Our image representation offers four advantages: (1) efficiency and compactness—concisely summarizing the 3D contextual information around an embolus in only three image channels, (2) consistency—automatically aligning the embolus in the 3-channel images according to the orientation of the affected vessel, (3) expandability—naturally supporting data augmentation for training CNNs, and (4) multi-view visualization—maximally revealing filling defects. To evaluate the effectiveness of VOIR for PE diagnosis, we use 121 CTPA datasets with a total of 326 emboli. We first compare VOIR with two other compact alternatives using six CNN architectures of varying depths and under varying amounts of labeled training data. Our experiments demonstrate that VOIR enables faster training of a higher-performing model compared to the other compact representations, even in the absence of deep architectures and large labeled training sets. Our experiments comparing VOIR with the 3D image representation further demonstrate that the 2D CNN trained with VOIR achieves a significant performance gain over the 3D CNNs. Our robustness analyses also show that the suggested PE CAD is robust to the choice of CT scanner machines and the physical

[1]  Mahdi Saadatmand-Tarzjan,et al.  A new dataset of computed-tomography angiography images for computer-aided detection of pulmonary embolism , 2018, Scientific Data.

[2]  Xiaohui Xie,et al.  DeepEM: Deep 3D ConvNets With EM For Weakly Supervised Pulmonary Nodule Detection , 2018, bioRxiv.

[3]  Abdel-Razzak M. Al-hinnawi,et al.  Collaboration between interactive three-dimensional visualization and computer aided detection of pulmonary embolism on computed tomography pulmonary angiography views , 2018, Radiological Physics and Technology.

[4]  Lei Zhang,et al.  Fine-Tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Aoxue Li,et al.  Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks , 2017, MICCAI.

[6]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[7]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[8]  Nima Tajbakhsh,et al.  Automated Polyp Detection in Colonoscopy Videos Using Shape and Context Information , 2016, IEEE Transactions on Medical Imaging.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Nima Tajbakhsh,et al.  Computer-Aided Pulmonary Embolism Detection Using a Novel Vessel-Aligned Multi-planar Image Representation and Convolutional Neural Networks , 2015, MICCAI.

[11]  Ronald M. Summers,et al.  Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation , 2015, IEEE Transactions on Medical Imaging.

[12]  Nima Tajbakhsh,et al.  Automatic polyp detection in colonoscopy videos using an ensemble of convolutional neural networks , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[13]  Ronald M. Summers,et al.  Deep convolutional networks for pancreas segmentation in CT imaging , 2015, Medical Imaging.

[14]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[15]  Ronald M. Summers,et al.  Detection of Sclerotic Spine Metastases via Random Aggregation of Deep Convolutional Neural Network Classifications , 2014, ArXiv.

[16]  Ronald M. Summers,et al.  A New 2.5D Representation for Lymph Node Detection Using Random Sets of Deep Convolutional Neural Network Observations , 2014, MICCAI.

[17]  Onur Osman,et al.  A novel method for pulmonary embolism detection in CTA images , 2014, Comput. Methods Programs Biomed..

[18]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[19]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[20]  Brian E. Chapman,et al.  Improving performance of computer-aided detection of pulmonary embolisms by incorporating a new pulmonary vascular-tree segmentation algorithm , 2012, Medical Imaging.

[21]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Brian E. Chapman,et al.  A Multistage Approach to Improve Performance of Computer-Aided Detection of Pulmonary Embolisms Depicted on CT Images: Preliminary Investigation , 2011, IEEE Transactions on Biomedical Engineering.

[23]  Gelareh Sadigh,et al.  Challenges, controversies, and hot topics in pulmonary embolism imaging. , 2011, AJR. American journal of roentgenology.

[24]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[25]  Anna Vilanova,et al.  Automatic Detection of Pulmonary Embolism in CTA Images , 2009, IEEE Transactions on Medical Imaging.

[26]  K. Marten,et al.  Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists , 2008, European Radiology.

[27]  M. Godoy,et al.  Computer-aided Detection of Pulmonary Embolism on CT Angiography: Initial Experience , 2007, Journal of thoracic imaging.

[28]  Maximilian Reiser,et al.  Clinical evaluation of a computer-aided diagnosis (CAD) prototype for the detection of pulmonary embolism. , 2007, Academic radiology.

[29]  Berkman Sahiner,et al.  Preliminary investigation of computer-aided detection of pulmonary embolism in three-dimensional computed tomography pulmonary angiography images. , 2005, Academic radiology.

[30]  Mel Herbert,et al.  The mortality of untreated pulmonary embolism in emergency department patients. , 2005, Annals of emergency medicine.

[31]  J. Fairfield,et al.  Toboggan contrast enhancement for contrast segmentation , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[32]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[33]  Francis L. Merat,et al.  Introduction to robotics: Mechanics and control , 1987, IEEE J. Robotics Autom..

[34]  Daniel Rueckert,et al.  Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II , 2017, Lecture Notes in Computer Science.