Cell segmentation methods for label-free contrast microscopy: review and comprehensive comparison

BackgroundBecause of its non-destructive nature, label-free imaging is an important strategy for studying biological processes. However, routine microscopic techniques like phase contrast or DIC suffer from shadow-cast artifacts making automatic segmentation challenging. The aim of this study was to compare the segmentation efficacy of published steps of segmentation work-flow (image reconstruction, foreground segmentation, cell detection (seed-point extraction) and cell (instance) segmentation) on a dataset of the same cells from multiple contrast microscopic modalities.ResultsWe built a collection of routines aimed at image segmentation of viable adherent cells grown on the culture dish acquired by phase contrast, differential interference contrast, Hoffman modulation contrast and quantitative phase imaging, and we performed a comprehensive comparison of available segmentation methods applicable for label-free data. We demonstrated that it is crucial to perform the image reconstruction step, enabling the use of segmentation methods originally not applicable on label-free images. Further we compared foreground segmentation methods (thresholding, feature-extraction, level-set, graph-cut, learning-based), seed-point extraction methods (Laplacian of Gaussians, radial symmetry and distance transform, iterative radial voting, maximally stable extremal region and learning-based) and single cell segmentation methods. We validated suitable set of methods for each microscopy modality and published them online.ConclusionsWe demonstrate that image reconstruction step allows the use of segmentation methods not originally intended for label-free imaging. In addition to the comprehensive comparison of methods, raw and reconstructed annotated data and Matlab codes are provided.

[1]  Yousef Al-Kofahi,et al.  Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images , 2010, IEEE Transactions on Biomedical Engineering.

[2]  Hui Kong,et al.  A Generalized Laplacian of Gaussian Filter for Blob Detection and Its Applications , 2013, IEEE Transactions on Cybernetics.

[3]  Alejandro F. Frangi,et al.  Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 , 2018, Lecture Notes in Computer Science.

[4]  Min Zhang,et al.  Small Blob Identification in Medical Images Using Regional Features From Optimum Scale , 2015, IEEE Transactions on Biomedical Engineering.

[5]  Zhuo Wang,et al.  Label-free intracellular transport measured by spatial light interference microscopy. , 2011, Journal of biomedical optics.

[6]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Erik H. W. Meijering,et al.  Cell Segmentation: 50 Years Down the Road [Life Sciences] , 2012, IEEE Signal Processing Magazine.

[8]  Olav Solgaard,et al.  Automated Cell Segmentation for Quantitative Phase Microscopy , 2018, IEEE Transactions on Medical Imaging.

[9]  Lóránd Kelemen,et al.  DIC image reconstruction using an energy minimization framework to visualize optical path length distribution , 2016, Scientific Reports.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  R. Barer.  Refractometry and interferometry of living cells. , 1957, Journal of the Optical Society of America.

[12]  Fabian Rudolf,et al.  Accurate cell segmentation in microscopy images using membrane patterns , 2014, Bioinform..

[13]  Jeffrey J. Rodriguez,et al.  Symmetry-based detection of nuclei in microscopy images , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[15]  Hervé Delingette,et al.  Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012 , 2012, Lecture Notes in Computer Science.

[16]  Claus Bahlmann,et al.  Fast radial symmetry detection under affine transformations , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Naveen C. Kumar,et al.  WORKSHOPS , 1993, 2022 Moratuwa Engineering Research Conference (MERCon).

[18]  E. Meijering.  Cell Segmentation : 50 Years Down the Road , 2012 .

[19]  Amit Gefen,et al.  A Method for Quick, Low-Cost Automated Confluency Measurements , 2011, Microscopy and Microanalysis.

[20]  Eugene W. Myers,et al.  Cell Detection with Star-convex Polygons , 2018, MICCAI.

[21]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[22]  Cédric Lhoussaine,et al.  Long-term tracking of budding yeast cells in brightfield microscopy: CellStar and the Evaluation Platform , 2017, Journal of The Royal Society Interface.

[23]  Pekka J. Toivanen,et al.  Shortest routes on varying height surfaces using gray-level distance transforms , 2005, Image Vis. Comput..

[24]  P Bajcsy,et al.  Empirical gradient threshold technique for automated segmentation across image modalities and cell lines , 2015, Journal of microscopy.

[25]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[26]  Nathalie Harder,et al.  An Objective Comparison of Cell Tracking Algorithms , 2017, Nature Methods.

[27]  Andrew Zisserman,et al.  Learning to Detect Cells Using Non-overlapping Extremal Regions , 2012, MICCAI.

[28]  Takeo Kanade,et al.  Restoring DIC Microscopy Images from Multiple Shear Directions , 2011, IPMI.

[29]  R. Barer.  Interference Microscopy and Mass Determination , 1952, Nature.

[30]  Luca Viganò,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2015, IWSEC 2015.

[31]  Alexey Shvets,et al.  TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation , 2018, Computer-Aided Analysis of Gastrointestinal Videos.

[32]  Euan A. Ashley,et al.  Deep Learning Automates the Quantitative Analysis of Individual Cells in Live-Cell Imaging Experiments , 2016, PLoS Comput. Biol..

[33]  Paul F. Whelan,et al.  A Novel Framework for Cellular Tracking and Mitosis Detection in Dense Phase Contrast Microscopy Images , 2013, IEEE Journal of Biomedical and Health Informatics.

[34]  Takeo Kanade,et al.  Understanding the phase contrast optics to restore artifact-free microscopy images for segmentation , 2012, Medical Image Anal..

[35]  Peter J O'Toole,et al.  Characterising live cell behaviour: Traditional label-free and quantitative phase imaging approaches. , 2017, The international journal of biochemistry & cell biology.

[36]  Adele P. Peskin,et al.  FogBank: a single cell segmentation across multiple cell lines and image modalities , 2014, BMC Bioinformatics.

[37]  Ross T. Whitaker,et al.  A Level-Set Approach to 3D Reconstruction from Range Data , 1998, International Journal of Computer Vision.

[38]  Jun Xu,et al.  Multi-Pass Adaptive Voting for Nuclei Detection in Histopathological Images , 2016, Scientific Reports.

[39]  Nathalie Harder,et al.  A benchmark for comparison of cell tracking algorithms , 2014, Bioinform..

[40]  Mrinal K. Mandal,et al.  Automatic Nuclei Detection Based on Generalized Laplacian of Gaussian Filters , 2017, IEEE Journal of Biomedical and Health Informatics.

[41]  K. Parvati,et al.  Image Segmentation Using Gray-Scale Morphology and Marker-Controlled Watershed Transformation , 2008 .

[42]  M. Omair Ahmad,et al.  A Method for Automatic Segmentation of Nuclei in Phase-Contrast Images Based on Intensity, Convexity and Texture , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[43]  Carl Duchesne,et al.  Selection and Tuning of a Fast and Simple Phase-Contrast Microscopy Image Segmentation Algorithm for Measuring Myoblast Growth Kinetics in an Automated Manner , 2013, Microscopy and Microanalysis.

[44]  Takeo Kanade,et al.  Phase Contrast Image Restoration via Dictionary Representation of Diffraction Patterns , 2012, MICCAI.

[45]  C Münzenmayer,et al.  Review of free software tools for image analysis of fluorescence cell micrographs , 2014, Journal of microscopy.

[46]  Carlos Ortiz-de-Solorzano,et al.  Segmentation of Touching Cell Nuclei Using a Two-Stage Graph Cut Model , 2009, SCIA.

[47]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[48]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[49]  Carolina Wählby,et al.  Feature Augmented Deep Neural Networks for Segmentation of Cells , 2016, ECCV Workshops.

[50]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[51]  Alexander Zelinsky,et al.  Fast Radial Symmetry for Detecting Points of Interest , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[53]  Qing Yang,et al.  Iterative Voting for Inference of Structural Saliency and Characterization of Subcellular Events , 2007, IEEE Transactions on Image Processing.

[54]  Péter Horváth,et al.  DIC Microscopy Image Reconstruction Using a Novel Variational Framework , 2015, 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[55]  Radim Chmelík,et al.  Coherence-controlled holographic microscope. , 2010, Optics express.

[56]  Kristian Bredies,et al.  An active-contour based algorithm for the automated segmentation of dense yeast populations on transmission microscopy images , 2011, Comput. Vis. Sci..

[57]  Lewis D. Griffin,et al.  Automated Method for the Rapid and Precise Estimation of Adherent Cell Culture Characteristics from Phase Contrast Microscopy Images , 2013, Biotechnology and bioengineering.

[58]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[59]  B. Heise,et al.  CELL SEGMENTATION USING LEVEL SET METHOD , 2007 .

[60]  Xiaobo Zhou,et al.  Integrating multi-scale blob/curvilinear detector techniques and multi-level sets for automated segmentation of stem cell images , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[61]  Lin Yang,et al.  Robust Segmentation of Overlapping Cells in Histopathology Specimens Using Parallel Seed Detection and Repulsive Level Set , 2012, IEEE Transactions on Biomedical Engineering.

[62]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[64]  Paul A. Wiggins,et al.  SuperSegger: robust image segmentation, analysis and lineage tracking of bacterial cells , 2016, Molecular Microbiology.

[65]  H. Sebastian Seung,et al.  Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification , 2017, Bioinform..

[66]  Lin Yang,et al.  Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review , 2016, IEEE Reviews in Biomedical Engineering.

[67]  Fabian J. Theis,et al.  fastER: a user‐friendly tool for ultrafast and robust cell segmentation in large‐scale microscopy , 2017, Bioinform..

[68]  R. Chmelík,et al.  Off-axis setup taking full advantage of incoherent illumination in coherence-controlled holographic microscope. , 2013, Optics express.