Learning Generative Models of Tissue Organization with Supervised GANs

A key step in understanding the spatial organization of cells and tissues is the ability to construct generative models that accurately reflect that organization. In this paper, we focus on building generative models of electron microscope (EM) images in which the positions of cell membranes and mitochondria have been densely annotated, and propose a two-stage procedure that produces realistic images using Generative Adversarial Networks (or GANs) in a supervised way. In the first stage, we synthesize a label "image" given a noise "image" as input, which then provides supervision for EM image synthesis in the second stage. The full model naturally generates label-image pairs. We show that accurate synthetic EM images are produced using assessment via (1) shape features and global statistics, (2) segmentation accuracies, and (3) user studies. We also demonstrate further improvements by enforcing a reconstruction loss on intermediate synthetic labels and thus unifying the two stages into one single end-to-end framework.

[1]  R. D. Hryciw,et al.  Traditional soil particle sphericity, roundness and surface roughness by computational geometry , 2015 .

[2]  H. Sebastian Seung,et al.  Recursive Training of 2D-3D Convolutional Networks for Neuronal Boundary Prediction , 2015, NIPS.

[3]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[4]  L. Loew,et al.  The Virtual Cell: a software environment for computational cell biology. , 2001, Trends in biotechnology.

[5]  Vladlen Koltun,et al.  Photographic Image Synthesis with Cascaded Refinement Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Wei Xu,et al.  Look and Think Twice: Capturing Top-Down Visual Attention with Feedback Convolutional Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Federico Vaggi,et al.  GANs for Biological Image Synthesis , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[9]  David Svoboda,et al.  Generation of 3D Digital Phantoms of Colon Tissue , 2011, ICIAR.

[10]  Gerhard Stephan,et al.  Segmented anisotropic ssTEM dataset of neural tissue , 2013 .

[11]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[12]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[14]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[15]  David Pfau,et al.  Unrolled Generative Adversarial Networks , 2016, ICLR.

[16]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[17]  Abhinav Gupta,et al.  Generative Image Modeling Using Style and Structure Adversarial Networks , 2016, ECCV.

[18]  Joachim M. Buhmann,et al.  Crowdsourcing the creation of image segmentation algorithms for connectomics , 2015, Front. Neuroanat..

[19]  Mark H. Ellisman,et al.  Neuron segmentation in electron microscopy images using partial differential equations , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[20]  Robert F. Murphy,et al.  A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of HeLa cells , 2001, Bioinform..

[21]  Michal Kozubek,et al.  Generation of digital phantoms of cell nuclei and simulation of image formation in 3D image cytometry , 2009, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[22]  Alexandros G. Dimakis,et al.  CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training , 2017, ICLR.

[23]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[24]  David Lopez-Paz,et al.  Revisiting Classifier Two-Sample Tests , 2016, ICLR.

[25]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[26]  Eric P. Xing,et al.  Structured Generative Adversarial Networks , 2017, NIPS.

[27]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[28]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[29]  Ting Zhao,et al.  Automated learning of generative models for subcellular location: Building blocks for systems biology , 2007, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[30]  Anatole Chessel,et al.  An Overview of data science uses in bioimage informatics. , 2017, Methods.

[31]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[33]  Yi Zhang,et al.  Do GANs actually learn the distribution? An empirical study , 2017, ArXiv.

[34]  Erik Meijering,et al.  Imagining the future of bioimage analysis , 2016, Nature Biotechnology.