Modeling Temporal Information of Mitotic for Mitotic Event Detection

Due to the enormous potential and influence that stem cells may have in regenerative medicine, there has been a rapidly growing interest in developing tools to analyze and characterize the behaviors of these cells in vitro. Among these behaviors, mitosis, or cell division, is very important because stem cells proliferate and renew themselves through mitosis. However, current automated systems for mitosis detection often require traditional computer vision technology and machine learning methods; automated mitosis detection and recognition are difficult to achieve and mainly rely on manual annotation. In this paper, we proposed an effective method to capture video-wide temporal information for automated mitosis detection and recognition, which is a nondestructive imaging modality, thereby allowing continuous monitoring of cells in culture. In this approach, we postulate that a function capable of ordering the frames of a video temporally well captures the evolution of the appearance within the video. We learn such ranking functions per video via a ranking machine and use the parameters of these functions as a new video representation. Here, we utilized the CNN model (VGG-16) and some classic low-level feature extraction methods (HOG, SIFT, and GIST) to extract low-level features for each frame. The proposed method is easy to interpret and implement, fast to compute and effective in recognizing mitosis events. In a comparison experiment, our approach significantly outperformed previous approaches in terms of both detection accuracy and computational efficiency. The data that we validate the proposed method with includes C3H10 mesenchymal and C2C12 myoblastic stem cell populations. Our approach achieves an F-score of 95.8 percent on the C2C12 dataset and an F-score of 95.3 percent on the C3H10 dataset. The results on both datasets outperform traditional mitosis recognition methods based on probability models. These experiments all demonstrate the significance of our approach.

[1]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[2]  Takeo Kanade,et al.  Mitosis sequence detection using hidden conditional random fields , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[3]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[4]  Jens Rittscher,et al.  Coupled Minimum-Cost Flow Cell Tracking , 2009, IPMI.

[5]  Yuting Su,et al.  HEp-2 cells Classification via clustered multi-task learning , 2016, Neurocomputing.

[6]  Yue Gao,et al.  Continuous Probability Distribution Prediction of Image Emotions via Multitask Shared Sparse Regression , 2017, IEEE Transactions on Multimedia.

[7]  Takeo Kanade,et al.  Spatiotemporal mitosis event detection in time-lapse phase contrast microscopy image sequences , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[8]  David A. Clausi,et al.  Automated Detection of Mitosis in Embryonic Tissues , 2007, Fourth Canadian Conference on Computer and Robot Vision (CRV '07).

[9]  M. Melamed,et al.  Recognition of cells in mitosis by flow cytofluormetry. , 1977, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[10]  Trevor Darrell,et al.  Hidden Conditional Random Fields , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yue Gao,et al.  Predicting Personalized Image Emotion Perceptions in Social Networks , 2018, IEEE Transactions on Affective Computing.

[12]  Michael Unser,et al.  A new hybrid Bayesian-variational particle filter with application to mitotic cell tracking , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[13]  Wenquan Feng,et al.  An architecture of optimised SIFT feature detection for an FPGA implementation of an image matcher , 2009, 2009 International Conference on Field-Programmable Technology.

[14]  Milan Sonka,et al.  Mitotic cell recognition with hidden Markov models , 2004, Medical Imaging: Image-Guided Procedures.

[15]  Andrew Zisserman,et al.  Discriminative Semi-Markov Models for automated mitotic phase labelling , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[16]  Dai Fei Elmer Ker,et al.  Tracking of Hematopoietic Stem Cells in Microscopy Images for Lineage Determination , 2011 .

[17]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[18]  Thomas Serre,et al.  A Biologically Inspired System for Action Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Mohan S. Kankanhalli,et al.  Hierarchical Clustering Multi-Task Learning for Joint Human Action Grouping and Recognition , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Wenhui Li,et al.  Cross-view action recognition by cross-domain learning , 2016, Image Vis. Comput..

[21]  Yuting Su,et al.  Sparse coding induced transfer learning for HEp-2 cell classification. , 2014, Bio-medical materials and engineering.

[22]  Xiaobo Zhou,et al.  A Novel Cell Segmentation Method and Cell Phase Identification Using Markov Model , 2009, IEEE Transactions on Information Technology in Biomedicine.

[23]  Milan Sonka,et al.  Cell Segmentation, Tracking, and Mitosis Detection Using Temporal Context , 2005, MICCAI.

[24]  Hans Burkhardt,et al.  Harmonic Filters for 3D Multichannel Data: Rotation Invariant Detection of Mitoses in Colorectal Cancer , 2010, IEEE Transactions on Medical Imaging.

[25]  Zhihui Lai,et al.  Multi-scale gist feature manifold for building recognition , 2011, Neurocomputing.

[26]  Eccles Ba,et al.  Automatic digital image analysis for identification of mitotic cells in synchronous mammalian cell cultures. , 1986 .

[27]  J. Alison Noble,et al.  Automated segmentation and alignment of mitotic nuclei for kymograph visualisation , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[28]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories , 2006 .

[29]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[31]  Arcot Sowmya,et al.  Cell tracking and mitosis detection using splitting flow networks in phase-contrast imaging , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[32]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[33]  Yuting Su,et al.  Sequential sparse representation for mitotic event recognition , 2013 .

[34]  Yuting Su,et al.  Nonnegative Mixed-Norm Convex Optimization for Mitotic Cell Detection in Phase Contrast Microscopy , 2013, Comput. Math. Methods Medicine.

[35]  Takeo Kanade,et al.  Reliably Tracking Partially Overlapping Neural Stem Cells in DIC Microscopy Image Sequences , 2009 .

[36]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[38]  Takeo Kanade,et al.  A Semi-Markov Model for Mitosis Segmentation in Time-Lapse Phase Contrast Microscopy Image Sequences of Stem Cell Populations , 2012, IEEE Transactions on Medical Imaging.

[39]  Takeo Kanade,et al.  Automated Mitosis Detection of Stem Cell Populations in Phase-Contrast Microscopy Images , 2011, IEEE Transactions on Medical Imaging.

[40]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[41]  O. Sertel,et al.  Computer-aided Prognosis of Neuroblastoma: Detection of mitosis and karyorrhexis cells in digitized histological images , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[42]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[43]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[44]  Takeo Kanade,et al.  Cell population tracking and lineage construction with spatiotemporal context , 2008, Medical Image Anal..

[45]  Yue Gao,et al.  Multimedia Social Event Detection in Microblog , 2015, MMM.

[46]  Yuting Su,et al.  Cell type-independent mitosis event detection via hidden-state conditional neural fields , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[47]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[48]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[50]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[51]  Stephen T. C. Wong,et al.  Mitosis cell identification with conditional random fields , 2007, 2007 IEEE/NIH Life Science Systems and Applications Workshop.

[52]  Amir Madany Mamlouk,et al.  From time lapse-data to genealogic trees: Using different contrast mechanisms to improve cell tracking , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[53]  Honglak Lee,et al.  Unsupervised feature learning for audio classification using convolutional deep belief networks , 2009, NIPS.

[54]  Mei Chen,et al.  Detection of mitosis within a stem cell population of high cell confluence in phase-contrast microscopy images , 2011, CVPR 2011.

[55]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .