Deep Learning for Robust Segmentation and Explainable Analysis of 3D and Dynamic Cardiac Images. (Apprentissage Profond pour la Segmentation Robuste et l'Analyse Explicable des Images Cardiaques Volumiques et Dynamiques)

Cardiac MRI is widely used by cardiologists as it allows extracting rich information from images. However, if done manually, the information extraction process is tedious and time-consuming. Given the advance of artificial intelligence, I develop deep learning methods to address the automation of several essential tasks on cardiac MRI analysis. First, I propose a method based on convolutional neural networks to perform cardiac segmentation on short axis MRI image stacks. In this method, since the prediction of a segmentation of a slice is dependent upon the already existing segmentation of an adjacent slice, 3D-consistency and robustness is explicitly enforced. Second, I develop a method to classify cardiac pathologies, with a novel deep learning approach to extract image-derived features to characterize the shape and motion of the heart. In particular, the classification model is explainable, simple and flexible. Last but not least, the same feature extraction method is applied to an exceptionally large dataset (UK Biobank). Unsupervised cluster analysis is then performed on the extracted features in search of their further relation with cardiac pathology characterization. To conclude, I discuss several possible extensions of my research.

[1]  Yuanyuan Wang,et al.  Left Ventricle Segmentation via Optical-Flow-Net from Short-Axis Cine MRI: Preserving the Temporal Coherence of Cardiac Motion , 2018, MICCAI.

[2]  Oscar Camara,et al.  A Radiomics Approach to Computer-Aided Diagnosis with Cardiac Cine-MRI , 2017, STACOM@MICCAI.

[3]  Scott D Flamm,et al.  Standardized image interpretation and post processing in cardiovascular magnetic resonance: Society for Cardiovascular Magnetic Resonance (SCMR) Board of Trustees Task Force on Standardized Post Processing , 2013, Journal of Cardiovascular Magnetic Resonance.

[4]  Arthur W. Toga,et al.  The Alzheimer's Disease Neuroimaging Initiative informatics core: A decade in review , 2015, Alzheimer's & Dementia.

[5]  Daniel Rueckert,et al.  Right ventricle segmentation from cardiac MRI: A collation study , 2015, Medical Image Anal..

[6]  Klaus H. Maier-Hein,et al.  Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges , 2017, Lecture Notes in Computer Science.

[7]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[8]  Xiaoguang Lu,et al.  Automatic Segmentation of the Myocardium in Cine MR Images Using Deformable Registration , 2011, STACOM.

[9]  Hamid Jafarkhani,et al.  A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI , 2015, Medical Image Anal..

[10]  Nicolas Duchateau,et al.  Infarct localization from myocardial deformation: Prediction and uncertainty quantification by regression from a low-dimensional space. , 2016, IEEE transactions on medical imaging.

[11]  M. Cerqueira,et al.  Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: A statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association , 2002, The international journal of cardiovascular imaging.

[12]  Alejandro F. Frangi,et al.  High Throughput Computation of Reference Ranges of Biventricular Cardiac Function on the UK Biobank Population Cohort , 2018, STACOM@MICCAI.

[13]  Caglar Senaras,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[14]  Sébastien Ourselin,et al.  Multi-atlas Propagation Whole Heart Segmentation from MRI and CTA Using a Local Normalised Correlation Coefficient Criterion , 2013, FIMH.

[15]  Dorin Comaniciu,et al.  Shaping the Future through Innovations: From Medical Imaging to Precision Medicine , 2016, Medical Image Anal..

[16]  Ben Glocker,et al.  Human-level CMR image analysis with deep fully convolutional networks , 2017, ArXiv.

[17]  Jan-Willem Romeijn,et al.  ‘All models are wrong...’: an introduction to model uncertainty , 2012 .

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Dorin Comaniciu,et al.  Cardiac Anchoring in MRI through Context Modeling , 2010, MICCAI.

[20]  Wufeng Xue,et al.  Full left ventricle quantification via deep multitask relationships learning , 2018, Medical Image Anal..

[21]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[22]  Ganapathy Krishnamurthi,et al.  Fully convolutional multi‐scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers , 2018, Medical Image Anal..

[23]  Pablo Lamata,et al.  Recurrent Fully Convolutional Neural Networks for Multi-slice MRI Cardiac Segmentation , 2016, RAMBO+HVSMR@MICCAI.

[24]  Yeonggul Jang,et al.  Automatic Segmentation of LV and RV in Cardiac MRI , 2017, STACOM@MICCAI.

[25]  M. Al-Shamsi,et al.  Addressing the physicians’ shortage in developing countries by accelerating and reforming the medical education: Is it possible? , 2017, Journal of advances in medical education & professionalism.

[26]  Ronald M. Summers,et al.  Medical Image Data and Datasets in the Era of Machine Learning—Whitepaper from the 2016 C-MIMI Meeting Dataset Session , 2017, Journal of Digital Imaging.

[27]  Nicholas Ayache,et al.  3-D Consistent and Robust Segmentation of Cardiac Images by Deep Learning With Spatial Propagation , 2018, IEEE Transactions on Medical Imaging.

[28]  Bertil Schmidt,et al.  $ν$-net: Deep Learning for Generalized Biventricular Cardiac Mass and Function Parameters , 2017, ArXiv.

[29]  Patrick van der Smagt,et al.  CNN-based Segmentation of Medical Imaging Data , 2017, ArXiv.

[30]  Douglas A. Reynolds,et al.  Gaussian Mixture Models , 2018, Encyclopedia of Biometrics.

[31]  Daniel Rueckert,et al.  Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences , 2018, MICCAI.

[32]  Phi Vu Tran,et al.  A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI , 2016, ArXiv.

[33]  Olivier Commowick,et al.  Imaging biomarkers in multiple Sclerosis: From image analysis to population imaging , 2016, Medical Image Anal..

[34]  Dorin Comaniciu,et al.  Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features , 2008, IEEE Transactions on Medical Imaging.

[35]  A A Young,et al.  4D modelling for rapid assessment of biventricular function in congenital heart disease , 2018, The International Journal of Cardiovascular Imaging.

[36]  Nico Karssemeijer,et al.  Using deep learning to segment breast and fibroglandular tissue in MRI volumes , 2017, Medical physics.

[37]  Daisuke Komura,et al.  Machine Learning Methods for Histopathological Image Analysis , 2017, Computational and structural biotechnology journal.

[38]  Patrick Clarysse,et al.  Estimation of cardiac motion in cine-MRI sequences by correlation transform optical flow of monogenic features distance , 2016, Physics in medicine and biology.

[39]  Iman Aganj,et al.  Unsupervised Medical Image Segmentation Based on the Local Center of Mass , 2018, Scientific Reports.

[40]  Hervé Delingette,et al.  Unsupervised Probabilistic Deformation Modeling for Robust Diffeomorphic Registration , 2018, DLMIA/ML-CDS@MICCAI.

[41]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[42]  Jürgen Weese,et al.  Four challenges in medical image analysis from an industrial perspective , 2016, Medical Image Anal..

[43]  Christian Biemann,et al.  What do we need to build explainable AI systems for the medical domain? , 2017, ArXiv.

[44]  Max A. Viergever,et al.  Automatic Segmentation and Disease Classification Using Cardiac Cine MR Images , 2017, STACOM@MICCAI.

[45]  James S. Duncan,et al.  Learning-based Regularization for Cardiac Strain Analysis with Ability for Domain Adaptation , 2018, ArXiv.

[46]  Ben Glocker,et al.  Learning clinically useful information from images: Past, present and future , 2016, Medical Image Anal..

[47]  Kilian M. Pohl,et al.  3D Motion Modeling and Reconstruction of Left Ventricle Wall in Cardiac MRI , 2017, FIMH.

[48]  George Lee,et al.  Image analysis and machine learning in digital pathology: Challenges and opportunities , 2016, Medical Image Anal..

[49]  R. Adelman,et al.  Caregiver burden: a clinical review. , 2014, JAMA.

[50]  C. Sudlow,et al.  Comparison of Sociodemographic and Health-Related Characteristics of UK Biobank Participants With Those of the General Population , 2017, American journal of epidemiology.

[51]  Daniel Rueckert,et al.  Machine Learning of Three-dimensional Right Ventricular Motion Enables Outcome Prediction in Pulmonary Hypertension: A Cardiac MR Imaging Study , 2017, Radiology.

[52]  Stefan K. Piechnik,et al.  Reference ranges for cardiac structure and function using cardiovascular magnetic resonance (CMR) in Caucasians from the UK Biobank population cohort , 2017, Journal of Cardiovascular Magnetic Resonance.

[53]  Charles Bouveyron,et al.  Class‐specific variable selection in high‐dimensional discriminant analysis through Bayesian Sparsity , 2018, Journal of Chemometrics.

[54]  Dinggang Shen,et al.  Anatomical Landmark Based Deep Feature Representation for MR Images in Brain Disease Diagnosis , 2018, IEEE Journal of Biomedical and Health Informatics.

[55]  Nicholas Ayache,et al.  Understanding the "Demon's Algorithm": 3D Non-rigid Registration by Gradient Descent , 1999, MICCAI.

[56]  Michael Mitzenmacher,et al.  Detecting Novel Associations in Large Data Sets , 2011, Science.

[57]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[58]  Ben Glocker,et al.  Disease prediction using graph convolutional networks: Application to Autism Spectrum Disorder and Alzheimer's disease , 2018, Medical Image Anal..

[59]  Cordelia Schmid,et al.  High-dimensional data clustering , 2006, Comput. Stat. Data Anal..

[60]  Hamid Jafarkhani,et al.  Automatic segmentation of the right ventricle from cardiac MRI using a learning‐based approach , 2017, Magnetic resonance in medicine.

[61]  Simona Moldovanu,et al.  Threshold selection for classification of MR brain images by clustering method , 2015 .

[62]  Denis Friboulet,et al.  Fast automatic myocardial segmentation in 4D cine CMR datasets , 2014, Medical Image Anal..

[63]  M. Avendi,et al.  Fully automatic segmentation of heart chambers in cardiac MRI using deep learning , 2016, Journal of Cardiovascular Magnetic Resonance.

[64]  C. Lee,et al.  Medical big data: promise and challenges , 2017, Kidney research and clinical practice.

[65]  Nicholas Ayache,et al.  Model-Based Generation of Large Databases of Cardiac Images: Synthesis of Pathological Cine MR Sequences From Real Healthy Cases , 2018, IEEE Transactions on Medical Imaging.

[66]  Hervé Delingette,et al.  Generation of Synthetic but Visually Realistic Time Series of Cardiac Images Combining a Biophysical Model and Clinical Images , 2013, IEEE Transactions on Medical Imaging.

[67]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[68]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[69]  Marleen de Bruijne,et al.  Machine learning approaches in medical image analysis: From detection to diagnosis , 2016, Medical Image Anal..

[70]  Khalid Raza,et al.  A Tour of Unsupervised Deep Learning for Medical Image Analysis , 2018, Current medical imaging.

[71]  Ganapathy Krishnamurthi,et al.  Densely Connected Fully Convolutional Network for Short-Axis Cardiac Cine MR Image Segmentation and Heart Diagnosis Using Random Forest , 2017, STACOM@MICCAI.

[72]  Alistair A. Young,et al.  Cardiac image modelling: Breadth and depth in heart disease , 2016, Medical Image Anal..

[73]  Maxime Sermesant,et al.  Statistical Atlases and Computational Models of the Heart. Imaging and Modelling Challenges , 2015, Lecture Notes in Computer Science.

[74]  Daniel Rueckert,et al.  Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image , 2018, MLMIR@MICCAI.

[75]  James S. Duncan,et al.  Flow Network Based Cardiac Motion Tracking Leveraging Learned Feature Matching , 2017, MICCAI.

[76]  Marc Pollefeys,et al.  An Exploration of 2D and 3D Deep Learning Techniques for Cardiac MR Image Segmentation , 2017, STACOM@MICCAI.

[77]  Daniel Forsberg,et al.  Detection and Labeling of Vertebrae in MR Images Using Deep Learning with Clinical Annotations as Training Data , 2017, Journal of Digital Imaging.

[78]  Yen-Wei Chen,et al.  HEp-2 Cell Classification Using K-Support Spatial Pooling in Deep CNNs , 2016, LABELS/DLMIA@MICCAI.

[79]  Imari Sato,et al.  Semi-supervised Learning for Biomedical Image Segmentation via Forest Oriented Super Pixels(Voxels) , 2017, MICCAI.

[80]  Lin Yang,et al.  Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation , 2016, NIPS.

[81]  Yong Fan,et al.  Non-rigid image registration using fully convolutional networks with deep self-supervision , 2017, ArXiv.

[82]  Hervé Delingette,et al.  Learning a Probabilistic Model for Diffeomorphic Registration , 2018, IEEE Transactions on Medical Imaging.

[83]  Alfonso Arellano,et al.  Medical Imaging Lesion Detection Based on Unified Gravitational Fuzzy Clustering , 2017, Journal of healthcare engineering.

[84]  Dorin Comaniciu,et al.  Learning-Based Detection and Tracking in Medical Imaging: A Probabilistic Approach , 2013 .

[85]  Josien P. W. Pluim,et al.  Not‐so‐supervised: A survey of semi‐supervised, multi‐instance, and transfer learning in medical image analysis , 2018, Medical Image Anal..

[86]  Nitin Singhal,et al.  Hybrid approach for automatic segmentation of fetal abdomen from ultrasound images using deep learning , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[87]  Alistair A. Young,et al.  Quantification of LV function and mass by cardiovascular magnetic resonance: multi-center variability and consensus contours , 2015, Journal of Cardiovascular Magnetic Resonance.

[88]  Joachim M. Buhmann,et al.  Glaucoma detection using entropy sampling and ensemble learning for automatic optic cup and disc segmentation , 2017, Comput. Medical Imaging Graph..

[89]  Hervé Delingette,et al.  Unsupervised shape and motion analysis of 3822 cardiac 4D MRIs of UK Biobank , 2019, ArXiv.

[90]  Jacques Bouaud,et al.  Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach , 2019, Artif. Intell. Medicine.

[91]  Hervé Delingette,et al.  3D Consistent Biventricular Myocardial Segmentation Using Deep Learning for Mesh Generation , 2018, ArXiv.

[92]  X. Pennec,et al.  Comparing algorithms for diffeomorphic registration: Stationary LDDMM and Diffeomorphic Demons , 2009 .

[93]  Dimitris N. Metaxas,et al.  Large-Scale medical image analytics: Recent methodologies, applications and Future directions , 2016, Medical Image Anal..

[94]  Xin Yang,et al.  Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? , 2018, IEEE Transactions on Medical Imaging.

[95]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[96]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[97]  Alejandro F. Frangi,et al.  Benchmarking framework for myocardial tracking and deformation algorithms: An open access database , 2013, Medical Image Anal..

[98]  Nicholas Ayache,et al.  LCC-Demons: A robust and accurate symmetric diffeomorphic registration algorithm , 2013, NeuroImage.

[99]  Milind E. Rane,et al.  Clustering Techniques for Brain Tumor Detection , 2014 .

[100]  Konstantinos Kamnitsas,et al.  Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation , 2017, IEEE Transactions on Medical Imaging.

[101]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[102]  G. Wainrib,et al.  Brain age prediction of healthy subjects on anatomic MRI with deep learning : going beyond with an “explainable AI” mindset , 2018, bioRxiv.

[103]  Hervé Delingette,et al.  Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow , 2018, Medical Image Anal..

[104]  Carmel Hayes,et al.  Fully-automated left ventricular mass and volume MRI analysis in the UK Biobank population cohort: evaluation of initial results , 2017, The International Journal of Cardiovascular Imaging.

[105]  Holger Roth,et al.  Unsupervised segmentation of 3D medical images based on clustering and deep representation learning , 2018, Medical Imaging.

[106]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[107]  Jan Rühaak,et al.  Highly accurate fast lung CT registration , 2013, Medical Imaging.

[108]  P. Matthews,et al.  UK Biobank’s cardiovascular magnetic resonance protocol , 2015, Journal of Cardiovascular Magnetic Resonance.