Training CNN Classifiers for Semantic Segmentation using Partially Annotated Images: with Application on Human Thigh and Calf MRI

Objective: Medical image datasets with pixel-level labels tend to have a limited number of organ or tissue label classes annotated, even when the images have wide anatomical coverage. With supervised learning, multiple classifiers are usually needed given these partially annotated datasets. In this work, we propose a set of strategies to train one single classifier in segmenting all label classes that are heterogeneously annotated across multiple datasets without moving into semi-supervised learning. Methods: Masks were first created from each label image through a process we termed presence masking. Three presence masking modes were evaluated, differing mainly in weightage assigned to the annotated and unannotated classes. These masks were then applied to the loss function during training to remove the influence of unannotated classes. Results: Evaluation against publicly available CT datasets shows that presence masking is a viable method for training class-generic classifiers. Our class-generic classifier can perform as well as multiple class-specific classifiers combined, while the training duration is similar to that required for one class-specific classifier. Furthermore, the class-generic classifier can outperform the class-specific classifiers when trained on smaller datasets. Finally, consistent results are observed from evaluations against human thigh and calf MRI datasets collected in-house. Conclusion: The evaluation outcomes show that presence masking is capable of significantly improving both training and inference efficiency across imaging modalities and anatomical regions. Improved performance may even be observed on small datasets. Significance: Presence masking strategies can reduce the computational resources and costs involved in manual medical image annotations. All codes are publicly available at this https URL.

[1]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[2]  Tobias Gass,et al.  Cloud-Based Evaluation of Anatomical Structure Segmentation and Landmark Detection Algorithms: VISCERAL Anatomy Benchmarks , 2016, IEEE Transactions on Medical Imaging.

[3]  Krista Vandenborne,et al.  Use of skeletal muscle MRI in diagnosis and monitoring disease progression in Duchenne muscular dystrophy. , 2012, Physical medicine and rehabilitation clinics of North America.

[4]  Marco Nolden,et al.  The Medical Imaging Interaction Toolkit , 2005, Medical Image Anal..

[5]  Song Wang,et al.  Three-Dimensional CT Image Segmentation by Combining 2D Fully Convolutional Network with 3D Majority Voting , 2016, LABELS/DLMIA@MICCAI.

[6]  Josien P. W. Pluim,et al.  Not‐so‐supervised: A survey of semi‐supervised, multi‐instance, and transfer learning in medical image analysis , 2018, Medical Image Anal..

[7]  Moi Hoon Yap,et al.  Automatic Segmentation of MRI Human Thigh Muscles: Combination of Reliable and Fast Framework Methods for Quadriceps, Femur and Marrow Segmentation , 2018, ICBET '18.

[8]  Yan Wang,et al.  Abdominal multi-organ segmentation with organ-attention networks and statistical fusion , 2018, Medical Image Anal..

[9]  Frédo Durand,et al.  Data augmentation using learned transforms for one-shot medical image segmentation , 2019, ArXiv.

[10]  Lonnie Lowery,et al.  Effect of creatine loading on anaerobic performance and skeletal muscle volume in NCAA Division I athletes. , 2002, Nutrition.

[11]  Ganapathy Krishnamurthi,et al.  Fully convolutional multi‐scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers , 2018, Medical Image Anal..

[12]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[13]  Vonda Wright,et al.  Chronic Exercise Preserves Lean Muscle Mass in Masters Athletes , 2011, The Physician and sportsmedicine.

[14]  Simon J. Doran,et al.  Stacked Autoencoders for Unsupervised Feature Learning and Multiple Organ Detection in a Pilot Study Using 4D Patient Data , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  Jing Yuan,et al.  HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation , 2018, IEEE Transactions on Medical Imaging.

[17]  Ronald M. Summers,et al.  Anatomy-specific classification of medical images using deep convolutional nets , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[18]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[19]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[20]  Daniel Rueckert,et al.  Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review , 2018, Medical Image Anal..

[21]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[22]  George Papandreou,et al.  Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation , 2015, ArXiv.

[23]  Fei-Fei Li,et al.  What's the Point: Semantic Segmentation with Point Supervision , 2015, ECCV.

[24]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[25]  Klaus H. Maier-Hein,et al.  nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation , 2018, Bildverarbeitung für die Medizin.

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  Konstantinos Kamnitsas,et al.  DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[28]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[29]  Marc Pollefeys,et al.  An Exploration of 2D and 3D Deep Learning Techniques for Cardiac MR Image Segmentation , 2017, STACOM@MICCAI.

[30]  M. Janier,et al.  Quantitative assessment of skeletal muscle degeneration in patients with myotonic dystrophy type 1 using MRI , 2012, Journal of magnetic resonance imaging : JMRI.

[31]  Fabio Gagliardi Cozman,et al.  Risks of Semi-Supervised Learning: How Unlabeled Data Can Degrade Performance of Generative Classifiers , 2006, Semi-Supervised Learning.

[32]  Naimul Mefraz Khan,et al.  A Novel Focal Tversky Loss Function With Improved Attention U-Net for Lesion Segmentation , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[33]  Aritra Ghosh,et al.  Robust Loss Functions under Label Noise for Deep Neural Networks , 2017, AAAI.

[34]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[35]  A. Tversky.  Features of Similarity , 1977 .

[36]  Ronald M. Summers,et al.  A large annotated medical image dataset for the development and evaluation of segmentation algorithms , 2019, ArXiv.

[37]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[38]  Scott T. Acton,et al.  Automated 3D muscle segmentation from MRI data using convolutional neural network , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[39]  Robert D. Nowak,et al.  Unlabeled data: Now it helps, now it doesn't , 2008, NIPS.

[40]  Shu Liao,et al.  Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition , 2016, IEEE Transactions on Medical Imaging.