Segmentation of Anatomical Layers and Artifacts in Intravascular Polarization Sensitive Optical Coherence Tomography Using Attending Physician and Boundary Cardinality Lost Terms

Intravascular ultrasound and optical coherence tomography are widely available for characterizing coronary stenoses and provide critical vessel parameters to optimize percutaneous intervention. Intravascular polarization-sensitive optical coherence tomography (PS-OCT) simultaneously provides high-resolution crosssectional images of vascular structures while also revealing preponderant tissue components such as collagen and smooth muscle and thereby enhances plaque characterization. Automated interpretation of these features promises to facilitate the objective clinical investigation of the natural history and significance of coronary atheromas. Here, we propose a convolutional neural network model, optimized using a new multi-term loss function, to classify the lumen, intima, and media layers in addition to the guidewire and plaque shadows. We demonstrate that our multi-class classification model outperforms state-of-theart methods in detecting the coronary anatomical layers. Furthermore, the proposed model segments two classes of common imaging artifacts and detects the anatomical layers within the thickened vessel wall regions that were excluded from analysis by other studies. The source code This work was supported by the National Institutes of Health NIBIB under Grants P41EB-015902 and P41EB-015903. Also, Mohammad Haft-Javaherian was supported by Bullock Postdoctoral Fellowship. This work was done partially using MIT-IBM Satori hardware resource (Corresponding author: Mohammad Haft-Javaherian). * Polina Golland and Brett E. Bouma have equal contributions. Mohammad Haft-Javaherian is with the Wellman Center for Photomedicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114 USA, and the Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology, Cambridge, MA 02142 USA (e-mail: haft@csail.mit.edu). Martin Villiger and Kenichiro Otsuka are with the Wellman Center for Photomedicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114 USA (e-mail: mvilliger@mgh.harvard.edu; kotsuka@mgh.harvard.edu). Joost Daemenis is with the Department of Interventional Cardiology, Thoraxcenter, Erasmus Medical Center, Rotterdam, The Netherlands (email: j.daemen@erasmusmc.nl). Peter Libby is with the Division of Cardiovascular Medicine, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA 02115 (e-mail: plibby@bwh.harvard.edu). Polina Golland is with the Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology, Cambridge, MA 02142 USA (e-mail: polina@csail.mit.edu). Brett E. Bouma is with the Wellman Center for Photomedicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114 USA, and the Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA 02142 USA (email: bouma@mgh.harvard.edu). and the trained model are publicly available at https:// github.com/mhaft/OCTseg.

[1]  A. Wahle,et al.  Early detection of cardiac allograft vasculopathy using highly automated 3-dimensional optical coherence tomography analysis. , 2018, The Journal of heart and lung transplantation : the official publication of the International Society for Heart Transplantation.

[2]  Elazer R. Edelman,et al.  Simultaneous Multi-Surface Fitting for Vessel Wall Layer Delineation , 2019, 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI).

[3]  Jeroen Eggermont,et al.  Fully automated side branch detection in intravascular optical coherence tomography pullback runs. , 2014, Biomedical optics express.

[4]  Zhi Chen,et al.  Quantitative 3D Analysis of Coronary Wall Morphology in Heart Transplant Patients: OCT‐Assessed Cardiac Allograft Vasculopathy Progression , 2018, Medical Image Anal..

[5]  Yih Miin Liew,et al.  Linear-regression convolutional neural network for fully automated coronary lumen segmentation in intravascular optical coherence tomography , 2017, Journal of biomedical optics.

[6]  Pedro A. Lemos,et al.  A robust fully automatic lumen segmentation method for in vivo intracoronary optical coherence tomography , 2016 .

[7]  Yiannis Kompatsiaris,et al.  ARC-OCT: Automatic detection of lumen border in intravascular OCT images , 2017, Comput. Methods Programs Biomed..

[8]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[9]  Stavros Tsantis,et al.  Automatic vessel lumen segmentation and stent strut detection in intravascular optical coherence tomography. , 2011, Medical physics.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Martin Villiger,et al.  Spectral binning for mitigation of polarization mode dispersion artifacts in catheter-based optical frequency domain imaging. , 2013, Optics express.

[12]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[13]  Sérgio Shiguemi Furuie,et al.  Automatic lumen segmentation in IVOCT images using binary morphological reconstruction , 2013, Biomedical engineering online.

[14]  Nassir Navab,et al.  Lumen Segmentation in Intravascular Optical Coherence Tomography Using Backscattering Tracked and Initialized Random Walks , 2016, IEEE Journal of Biomedical and Health Informatics.

[15]  Anil K. Jain,et al.  A modified Hausdorff distance for object matching , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[16]  Theo van Walsum,et al.  Contour segmentation of the intima, media, and adventitia layers in intracoronary OCT images: application to fully automatic detection of healthy wall regions , 2017, International Journal of Computer Assisted Radiology and Surgery.

[17]  G Tearney,et al.  Visualization of tissue prolapse between coronary stent struts by optical coherence tomography: comparison with intravascular ultrasound. , 2001, Circulation.

[18]  Qinye Yin,et al.  Automatic Lumen Segmentation in Intravascular Optical Coherence Tomography Images Using Level Set , 2017, Comput. Math. Methods Medicine.

[19]  Pedro A. Lemos,et al.  A bifurcation identifier for IV-OCT using orthogonal least squares and supervised machine learning , 2015, Comput. Medical Imaging Graph..

[20]  B. Bouma,et al.  Intravascular Polarimetry for Tissue Characterization of Coronary Atherosclerosis , 2019, Circulation reports.

[21]  Milan Sonka,et al.  Fast and memory-efficient LOGISMOS graph search for intraretinal layer segmentation of 3D macular OCT scans , 2015, Medical Imaging.

[22]  Elazer R. Edelman,et al.  A Mechanical Approach for Smooth Surface Fitting to Delineate Vessel Walls in Optical Coherence Tomography Images , 2019, IEEE Transactions on Medical Imaging.

[23]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[24]  Andrew M. Rollins,et al.  Automatic segmentation of intravascular optical coherence tomography images for facilitating quantitative diagnosis of atherosclerosis , 2011, BiOS.

[25]  V. Fuster,et al.  Histopathologic characteristics of atherosclerotic coronary disease and implications of the findings for the invasive and noninvasive detection of vulnerable plaques. , 2013, Journal of the American College of Cardiology.

[26]  Luca Mainardi,et al.  Automatic segmentation of optical coherence tomography pullbacks of coronary arteries treated with bioresorbable vascular scaffolds: Application to hemodynamics modeling , 2019, PloS one.

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  Chen Zhi,et al.  Donor specific anti-HLA antibodies and cardiac allograft vasculopathy: A prospective study using highly automated 3-D optical coherence tomography analysis. , 2020, Transplant immunology.

[29]  F. Cheriet,et al.  FULLY AUTOMATIC ARTIFICIAL INTELLIGENCE DIAGNOSTIC MODEL OF CORONARY ARTERY LESIONS USING OCT IMAGING , 2019, Canadian Journal of Cardiology.

[30]  Eloisa Arbustini,et al.  Sources of error and interpretation of plaque morphology by optical coherence tomography. , 2006, The American journal of cardiology.

[31]  David Wilson,et al.  Coronary calcification segmentation in intravascular OCT images using deep learning: application to calcification scoring , 2019, Journal of medical imaging.

[32]  Daniel Rueckert,et al.  Automatical vessel wall detection in intravascular coronary OCT , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[33]  W. Desmet,et al.  Automatic segmentation of in-vivo intra-coronary optical coherence tomography images to assess stent strut apposition and coverage , 2012, The International Journal of Cardiovascular Imaging.

[34]  Arsalan Akbar,et al.  Automated accurate lumen segmentation using L-mode interpolation for three-dimensional intravascular optical coherence tomography. , 2019, Biomedical optics express.

[35]  B. Bouma,et al.  Intravascular Polarimetry: Clinical Translation and Future Applications of Catheter-Based Polarization Sensitive Optical Frequency Domain Imaging , 2020, Frontiers in Cardiovascular Medicine.

[36]  Olivier Morel,et al.  Optical Coherence Tomography to Optimize Results of Percutaneous Coronary Intervention in Patients with Non–ST-Elevation Acute Coronary Syndrome: Results of the Multicenter, Randomized DOCTORS Study (Does Optical Coherence Tomography Optimize Results of Stenting) , 2016, Circulation.

[37]  Vladislav N. Zimin,et al.  Comparison of stent expansion using a volumetric versus the conventional method through optical coherence tomography in an all-comers population. , 2020, Cardiovascular revascularization medicine : including molecular interventions.

[38]  Pavel V Hushcha,et al.  Machine Learning Approaches in Cardiovascular Imaging , 2017, Circulation. Cardiovascular imaging.

[39]  Francesca N. Delling,et al.  Heart Disease and Stroke Statistics—2020 Update: A Report From the American Heart Association , 2020, Circulation.

[40]  Geraint Rees,et al.  Clinically applicable deep learning for diagnosis and referral in retinal disease , 2018, Nature Medicine.

[41]  Cilwyn S Braganza,et al.  Constrained polarization evolution simplifies depth-resolved retardation measurements with polarization-sensitive optical coherence tomography. , 2019, Biomedical optics express.

[42]  Ahmed Hosny,et al.  Artificial intelligence in radiology , 2018, Nature Reviews Cancer.

[43]  C. Villani Optimal Transport: Old and New , 2008 .

[44]  Seyed Kamaleddin Mostafavi Yazdi,et al.  A novel automated lumen segmentation and classification algorithm for detection of irregular protrusion after stents deployment , 2020, The international journal of medical robotics + computer assisted surgery : MRCAS.

[45]  Tiegen Liu,et al.  Automatic lumen segmentation using uniqueness of vascular connected region for intravascular optical coherence tomography , 2021, Journal of biophotonics.

[46]  Hongki Yoo,et al.  Automated detection of vessel lumen and stent struts in intravascular optical coherence tomography to evaluate stent apposition and neointimal coverage. , 2016, Medical physics.

[47]  Shiju Joseph,et al.  Automatic segmentation of coronary morphology using transmittance-based lumen intensity-enhanced intravascular optical coherence tomography images and applying a localized level-set-based active contour method , 2016, Journal of medical imaging.

[48]  Usama Bilal,et al.  Challenges and opportunities for cardiovascular disease prevention. , 2011, The American journal of medicine.

[49]  B. Bouma,et al.  Polarimetric Signatures of Vascular Tissue Response to Drug-Eluting Stent Implantation in Patients. , 2020, JACC. Cardiovascular imaging.

[50]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[51]  F. Cheriet,et al.  Characterization of coronary artery pathological formations from OCT imaging using deep learning. , 2018, Biomedical optics express.

[52]  Brett E Bouma,et al.  Measurement of collagen and smooth muscle cell content in atherosclerotic plaques using polarization-sensitive optical coherence tomography. , 2007, Journal of the American College of Cardiology.

[53]  Brett E. Bouma,et al.  Coronary Plaque Microstructure and Composition Modify Optical Polarization , 2017, JACC. Cardiovascular imaging.

[54]  Steven E. Nissen,et al.  High Prevalence of Coronary Atherosclerosis in Asymptomatic Teenagers and Young Adults: Evidence From Intravascular Ultrasound , 2001, Circulation.

[55]  Brett E. Bouma,et al.  Repeatability Assessment of Intravascular Polarimetry in Patients , 2018, IEEE Transactions on Medical Imaging.

[56]  Masataka Nakano,et al.  Acute Coronary Events , 2012, Circulation.