Neuro-fuzzy patch-wise R-CNN for multiple sclerosis segmentation

The segmentation of the lesion plays a core role in diagnosis and monitoring of multiple sclerosis (MS). Magnetic resonance imaging (MRI) is the most frequent image modality used to evaluate such lesions. Because of the massive amount of data, manual segmentation cannot be achieved within a sensible time that restricts the usage of accurate quantitative measurement in clinical practice. Therefore, the need for effective automated segmentation techniques is critical. However, a large spatial variability between the structure of brain lesions makes it more challenging. Recently, convolutional neural network (CNN), in particular, the region-based CNN (R-CNN), have attained tremendous progress within the field of object recognition because of its ability to learn and represent features. CNN has proven a last-breaking performance in various fields, such as object recognition, and has also gained more attention in brain imaging, especially in tissue and brain segmentation. In this paper, an automated technique for MS lesion segmentation is proposed, which is built on a 3D patch-wise R-CNN. The proposed system includes two stages: first, segmenting MS lesions in T2-w and FLAIR sequences using R-CNN, then an adaptive neuro-fuzzy inference system (ANFIS) is applied to fuse the results of the two modalities. To evaluate the performance of the proposed method, the public MICCAI2008 MS challenge dataset is employed to segment MS lesions. The experimental results show competitive results of the proposed method compared with the state-of-the-art MS lesion segmentation methods with an average total score of 83.25 and an average sensitivity of 61.8 % on the MICCAI2008 testing set. Graphical Abstract The proposed system overview. First, the input of two modalities FLAIR and T2 are pre-processed to remove the skull and correct the bias field. Then 3D patches for lesion and non-lesion tissues are extracted and fed to R-CNN. Each R-CNN produces a probability map of the segmentation result that provides to ANFIS to fuse the results and obtain the final MS lesion segmentation. The MS lesions are shown on a pre-processed FLAIR image.

[1]  Alan C. Evans,et al.  A nonparametric method for automatic correction of intensity nonuniformity in MRI data , 1998, IEEE Transactions on Medical Imaging.

[2]  Alex Rovira,et al.  Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach , 2017, NeuroImage.

[3]  Hayit Greenspan,et al.  LESION DETECTION IN NOISY MR BRAIN IMAGES USING CONSTRAINED GMM AND ACTIVE CONTOURS , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[4]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xianghua Xie,et al.  Minimum S-Excess Graph for Segmenting and Tracking Multiple Borders with HMM , 2015, MICCAI.

[6]  Simon K. Warfield,et al.  A Model of Population and Subject (MOPS) Intensities With Application to Multiple Sclerosis Lesion Segmentation , 2015, IEEE Transactions on Medical Imaging.

[7]  Xianghua Xie,et al.  Coupled s‐excess HMM for vessel border tracking and segmentation , 2019, International journal for numerical methods in biomedical engineering.

[8]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[9]  A. Oliver,et al.  A toolbox for multiple sclerosis lesion segmentation , 2015, Neuroradiology.

[10]  Christos Davatzikos,et al.  Computer-assisted Segmentation of White Matter Lesions in 3d Mr Images Using Support Vector Machine 1 , 2022 .

[11]  Antonio Cerasa,et al.  A Cellular Neural Network methodology for the automated segmentation of multiple sclerosis lesions , 2012, Journal of Neuroscience Methods.

[12]  Bostjan Likar,et al.  A Review of Methods for Correction of Intensity Inhomogeneity in MRI , 2007, IEEE Transactions on Medical Imaging.

[13]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[14]  Mohammad Havaei,et al.  HeMIS: Hetero-Modal Image Segmentation , 2016, MICCAI.

[15]  Christian Barillot,et al.  Classification of multiple sclerosis lesions using adaptive dictionary learning , 2015, Comput. Medical Imaging Graph..

[16]  Bernhard Hemmer,et al.  An automated tool for detection of FLAIR-hyperintense white-matter lesions in Multiple Sclerosis , 2012, NeuroImage.

[17]  Koenraad Van Leemput,et al.  Automated segmentation of multiple sclerosis lesions by model outlier detection , 2001, IEEE Transactions on Medical Imaging.

[18]  Xianghua Xie,et al.  Automatic segmentation of cross-sectional coronary arterial images , 2017, Comput. Vis. Image Underst..

[19]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[20]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[21]  D. Louis Collins,et al.  Trimmed-Likelihood Estimation for Focal Lesions and Tissue Segmentation in Multisequence MRI for Multiple Sclerosis , 2011, IEEE Transactions on Medical Imaging.

[22]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[23]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[24]  U. Rajendra Acharya,et al.  Application of deep transfer learning for automated brain abnormality classification using MR images , 2019, Cognitive Systems Research.

[25]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[26]  Hayit Greenspan,et al.  Multi-view longitudinal CNN for multiple sclerosis lesion segmentation , 2017, Eng. Appl. Artif. Intell..

[27]  Hayit Greenspan,et al.  Patch-Based Segmentation with Spatial Consistency: Application to MS Lesions in Brain MRI , 2016, Int. J. Biomed. Imaging.

[28]  Olivier Clatz,et al.  Spatial Decision Forests for MS Lesion Segmentation in Multi-Channel MR Images , 2010, MICCAI.

[29]  Patrick M. Pilarski,et al.  First steps towards an intelligent laser welding architecture using deep neural networks and reinforcement learning , 2014 .

[30]  Saurabh Jain,et al.  Automatic segmentation and volumetry of multiple sclerosis brain lesions from MR images , 2015, NeuroImage: Clinical.

[31]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[32]  Peter A. Calabresi,et al.  A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions , 2010, NeuroImage.

[33]  Daniel Rueckert,et al.  Multiple Sclerosis Lesion Segmentation Using Dictionary Learning and Sparse Coding , 2013, MICCAI.

[34]  Ying Wu,et al.  Automated segmentation of multiple sclerosis lesion subtypes with multichannel MRI , 2006, NeuroImage.

[35]  D. Louis Collins,et al.  Probabilistic Multiple Sclerosis Lesion Classification Based on Modeling Regional Intensity Variability and Local Neighborhood Information , 2015, IEEE Transactions on Biomedical Engineering.

[36]  D. Louis Collins,et al.  Rotation-invariant multi-contrast non-local means for MS lesion segmentation , 2015, NeuroImage: Clinical.

[37]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[38]  Ninon Burgos,et al.  New advances in the Clinica software platform for clinical neuroimaging studies , 2019 .

[39]  Doina Precup,et al.  IMaGe: Iterative Multilevel Probabilistic Graphical Model for Detection and Segmentation of Multiple Sclerosis Lesions in Brain MRI , 2015, IPMI.

[40]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Jianxiong Xiao,et al.  R-CNN for Small Object Detection , 2016, ACCV.

[44]  Nassir Navab,et al.  Semi-supervised Deep Learning for Fully Convolutional Networks , 2017, MICCAI.

[45]  Joseph Ross Mitchell,et al.  Segmentation of multiple sclerosis lesions using support vector machines , 2003, SPIE Medical Imaging.