Toward automatic phenotyping of developing embryos from videos

We describe a trainable system for analyzing videos of developing C. elegans embryos. The system automatically detects, segments, and locates cells and nuclei in microscopic images. The system was designed as the central component of a fully automated phenotyping system. The system contains three modules 1) a convolutional network trained to classify each pixel into five categories: cell wall, cytoplasm, nucleus membrane, nucleus, outside medium; 2) an energy-based model, which cleans up the output of the convolutional network by learning local consistency constraints that must be satisfied by label images; 3) a set of elastic models of the embryo at various stages of development that are matched to the label images.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  Steven W. Zucker,et al.  On the Foundations of Relaxation Labeling Processes , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[4]  Geoffrey E. Hinton,et al.  Adaptive Elastic Models for Hand-Printed Character Recognition , 1991, NIPS.

[5]  Joost N. Kok,et al.  Motion planning using a colored Kohonen network , 1993 .

[6]  R. Vaillant,et al.  An original approach for the localization of objects in images , 1993 .

[7]  Jorge S. Marques,et al.  A common framework for snakes and Kohonen networks , 1993, Neural Networks for Signal Processing III - Proceedings of the 1993 IEEE-SP Workshop.

[8]  Marcello Pelillo,et al.  Learning Compatibility Coefficients for Relaxation Labeling Processes , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  R. Vaillant,et al.  Original approach for the localisation of objects in images , 1994 .

[10]  John C. Platt,et al.  A Convolutional Neural Network Hand Tracker , 1994, NIPS.

[11]  Yoshua Bengio,et al.  Word normalization for on-line handwritten word recognition , 1994 .

[12]  Yoshua Bengio,et al.  Word normalization for online handwritten word recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[13]  Stan Z. Li,et al.  Markov Random Field Modeling in Computer Vision , 1995, Computer Science Workbench.

[14]  Geoffrey E. Hinton,et al.  Using Generative Models for Handwritten Digit Recognition , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Demetri Terzopoulos,et al.  Deformable models in medical image analysis: a survey , 1996, Medical Image Anal..

[16]  Anil K. Jain,et al.  Object Matching Using Deformable Templates , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  L. V. van Vliet,et al.  Reconstruction of optical pathlength distributions from images obtained by a wide‐field differential interference contrast microscope , 1997, Journal of microscopy.

[18]  Jerry L. Prince,et al.  Gradient vector flow: a new external force for snakes , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[20]  Mark H. Overmars,et al.  Motion Planning with Complete Knowledge Using a Colored Som , 1997, Int. J. Neural Syst..

[21]  J. Berg.  Genome sequence of the nematode C. elegans: a platform for investigating biology. , 1998, Science.

[22]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[23]  Andrew Smith.  Genome sequence of the nematode C-elegans: A platform for investigating biology , 1998 .

[24]  A. Fire,et al.  Potent and specific genetic interference by double-stranded RNA in Caenorhabditis elegans , 1998, Nature.

[25]  M V Boland,et al.  Automated recognition of patterns characteristic of subcellular structures in fluorescence microscopy images. , 1998, Cytometry.

[26]  Yasuda,et al.  Towards Automatic Construction of Cell-Lineage of C. elegans from Nomarski DIC Microscope Images. , 1999, Genome informatics. Workshop on Genome Informatics.

[27]  Sebastian A. Leidel,et al.  Functional genomic analysis of cell division in C. elegans using RNAi of genes on chromosome III , 2000, Nature.

[28]  L. Stein,et al.  RNAi analysis of genes expressed in the ovary of Caenorhabditis elegans , 2000, Current Biology.

[29]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[30]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[32]  P. Zipperlen,et al.  Roles for 147 embryonic lethal genes on C.elegans chromosome I identified by RNA interference and video microscopy , 2001, The EMBO journal.

[33]  F. Piano,et al.  Gene Clustering Based on RNAi Phenotypes of Ovary-Enriched Genes in C. elegans , 2002, Current Biology.

[34]  Martial Hebert,et al.  Discriminative Fields for Modeling Spatial Dependencies in Natural Images , 2003, NIPS.

[35]  Yee Whye Teh,et al.  Energy-Based Models for Sparse Overcomplete Representations , 2003, J. Mach. Learn. Res..

[36]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Christophe Garcia,et al.  Convolutional face finder: a neural architecture for fast and robust face detection , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Yann LeCun,et al.  Synergistic Face Detection and Pose Estimation with Energy-Based Models , 2004, J. Mach. Learn. Res..

[39]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[40]  Robert F Murphy,et al.  From quantitative microscopy to automated image understanding. , 2004, Journal of biomedical optics.

[41]  C. Conrad,et al.  Automatic identification of subcellular phenotypes on human cell arrays. , 2004, Genome research.

[42]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[43]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  Yann LeCun,et al.  Loss Functions for Discriminative Training of Energy-Based Models , 2005, AISTATS.