Automatic improvement of deep learning-based cell segmentation in time-lapse microscopy by neural architecture search

Abstract Motivation Live cell segmentation is a crucial step in biological image analysis and is also a challenging task because time-lapse microscopy cell sequences usually exhibit complex spatial structures and complicated temporal behaviors. In recent years, numerous deep learning-based methods have been proposed to tackle this task and obtained promising results. However, designing a network with excellent performance requires professional knowledge and expertise and is very time-consuming and labor-intensive. Recently emerged neural architecture search (NAS) methods hold great promise in eliminating these disadvantages, because they can automatically search an optimal network for the task. Results We propose a novel NAS-based solution for deep learning-based cell segmentation in time-lapse microscopy images. Different from current NAS methods, we propose (i) jointly searching non-repeatable micro architectures to construct the macro network for exploring greater NAS potential and better performance and (ii) defining a specific search space suitable for the live cell segmentation task, including the incorporation of a convolutional long short-term memory network for exploring the temporal information in time-lapse sequences. Comprehensive evaluations on the 2D datasets from the cell tracking challenge demonstrate the competitiveness of the proposed method compared to the state of the art. The experimental results show that the method is capable of achieving more consistent top performance across all ten datasets than the other challenge methods. Availabilityand implementation The executable files of the proposed method as well as configurations for each dataset used in the presented experiments will be available for non-commercial purposes from https://github.com/291498346/nas_cellseg. Supplementary information Supplementary data are available at Bioinformatics online.

[1]  Tammy Riklin-Raviv,et al.  Microscopy Cell Segmentation Via Convolutional LSTM Networks , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[2]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[3]  Song Han,et al.  ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware , 2018, ICLR.

[4]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[5]  Kalyanmoy Deb,et al.  NSGA-Net: neural architecture search using multi-objective genetic algorithm , 2018, GECCO.

[6]  Yiming Yang,et al.  DARTS: Differentiable Architecture Search , 2018, ICLR.

[7]  Daguang Xu,et al.  V-NAS: Neural Architecture Search for Volumetric Medical Image Segmentation , 2019, 2019 International Conference on 3D Vision (3DV).

[8]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Tianming Liu,et al.  Neural Architecture Search for Optimizing Deep Belief Network Models of fMRI Data , 2019, MMMI@MICCAI.

[10]  Theodore Lim,et al.  SMASH: One-Shot Model Architecture Search through HyperNetworks , 2017, ICLR.

[11]  Mirabela Rusu,et al.  A deep learning-based algorithm for 2-D cell segmentation in microscopy images , 2018, BMC Bioinformatics.

[12]  Yujie Li,et al.  NAS-Unet: Neural Architecture Search for Medical Image Segmentation , 2019, IEEE Access.

[13]  Jens Petersen,et al.  nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation , 2020, Nature Methods.

[14]  Daguang Xu,et al.  Searching Learning Strategy with Reinforcement Learning for 3D Medical Image Segmentation , 2019, MICCAI.

[15]  Ulas Bagci,et al.  Automatically Designing CNN Architectures for Medical Image Segmentation , 2018, MLMI@MICCAI.

[16]  Lassi Paavolainen,et al.  nucleAIzer: A Parameter-free Deep Learning Framework for Nucleus Segmentation Using Image Style Transfer , 2020, Cell systems.

[17]  Eric P. Xing,et al.  Neural Architecture Search for Adversarial Medical Image Segmentation , 2019, MICCAI.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Lin Yang,et al.  Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review , 2016, IEEE Reviews in Biomedical Engineering.

[20]  Alan L. Yuille,et al.  Genetic CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Chuanbo Huang,et al.  Segmentation of Cell Images Based on Improved Deep Learning Approach , 2020, IEEE Access.

[22]  Feifan Wang,et al.  Neural Architecture Search for Gliomas Segmentation on Multimodal Magnetic Resonance Imaging , 2020, ArXiv.

[23]  Romuere Rôdrigues Veloso e Silva,et al.  Deep learning for cell image segmentation and ranking , 2019, Comput. Medical Imaging Graph..

[24]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[25]  Vijay Vasudevan,et al.  Learning Transferable Architectures for Scalable Image Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Li Fei-Fei,et al.  Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Nathalie Harder,et al.  An Objective Comparison of Cell Tracking Algorithms , 2017, Nature Methods.

[28]  Yong Yu,et al.  Efficient Architecture Search by Network Transformation , 2017, AAAI.

[29]  Fabian Rudolf,et al.  Accurate cell segmentation in microscopy images using membrane patterns , 2014, Bioinform..

[30]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Taesup Kim,et al.  Scalable Neural Architecture Search for 3D Medical Image Segmentation , 2019, MICCAI.

[32]  Erik H. W. Meijering,et al.  Neural Architecture Search for Microscopy Cell Segmentation , 2020, MLMI@MICCAI.

[33]  Thomas Brox,et al.  Understanding and Robustifying Differentiable Architecture Search , 2020, ICLR.

[34]  Lars Kotthoff,et al.  Automated Machine Learning: Methods, Systems, Challenges , 2019, The Springer Series on Challenges in Machine Learning.

[35]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Feixiao Long,et al.  Microscopy cell nuclei segmentation with enhanced U-Net , 2020, BMC Bioinformatics.

[37]  Mohsen Rezvani,et al.  ResBCDU-Net: A Deep Learning Framework for Lung CT Image Segmentation , 2021, Sensors.

[38]  Weiwen Jiang,et al.  MS-NAS: Multi-Scale Neural Architecture Search for Medical Image Segmentation , 2020, MICCAI.

[39]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[40]  Fusheng Wang,et al.  Automated cell segmentation with 3D fluorescence microscopy images , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[41]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Erik H. W. Meijering,et al.  Cell Segmentation: 50 Years Down the Road [Life Sciences] , 2012, IEEE Signal Processing Magazine.