GANs for Biological Image Synthesis

In this paper, we propose a novel application of Generative Adversarial Networks (GAN) to the synthesis of cells imaged by fluorescence microscopy. Compared to natural images, cells tend to have a simpler and more geometric global structure that facilitates image generation. However, the correlation between the spatial pattern of different fluorescent proteins reflects important biological functions, and synthesized images have to capture these relationships to be relevant for biological applications. We adapt GANs to the task at hand and propose new models with casual dependencies between image channels that can generate multichannel images, which would be impossible to obtain experimentally. We evaluate our approach using two independent techniques and compare it against sensible baselines. Finally, we demonstrate that by interpolating across the latent space we can mimic the known changes in protein localization that occur through time during the cell cycle, allowing us to predict temporal evolution from static images.

[1]  Aaron C. Courville,et al.  Adversarially Learned Inference , 2016, ICLR.

[2]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[3]  Jascha Sohl-Dickstein,et al.  Improved generator objectives for GANs , 2016, ArXiv.

[4]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[6]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[7]  Thomas Brox,et al.  Learning to Generate Chairs, Tables and Cars with Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[9]  R. Tsien,et al.  Improved monomeric red, orange and yellow fluorescent proteins derived from Discosoma sp. red fluorescent protein , 2004, Nature Biotechnology.

[10]  Luke de Oliveira,et al.  Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis , 2017, Computing and Software for Big Science.

[11]  M. Balasubramanian,et al.  Meiotic actin rings are essential for proper sporulation in fission yeast , 2012, Journal of Cell Science.

[12]  Ruslan Salakhutdinov,et al.  On the Quantitative Analysis of Decoder-Based Generative Models , 2016, ICLR.

[13]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  M. Chalfie GREEN FLUORESCENT PROTEIN , 1995, Photochemistry and photobiology.

[15]  M. Chalfie,et al.  Green fluorescent protein as a marker for gene expression. , 1994, Science.

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Tom White,et al.  Sampling Generative Networks: Notes on a Few Effective Techniques , 2016, ArXiv.

[18]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[19]  Yann LeCun,et al.  Energy-based Generative Adversarial Networks , 2016, ICLR.

[20]  Anatole Chessel,et al.  An Overview of data science uses in bioimage informatics. , 2017, Methods.

[21]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[23]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[24]  Sophie G. Martin,et al.  Cell polarization in budding and fission yeasts. , 2014, FEMS microbiology reviews.

[25]  David Pfau,et al.  Unrolled Generative Adversarial Networks , 2016, ICLR.

[26]  Abhinav Gupta,et al.  Generative Image Modeling Using Style and Structure Adversarial Networks , 2016, ECCV.

[27]  Anne E Carpenter,et al.  Annotated high-throughput microscopy image sets for validation , 2012, Nature Methods.

[28]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[29]  Erik Meijering,et al.  Imagining the future of bioimage analysis , 2016, Nature Biotechnology.

[30]  Chuan Li,et al.  Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks , 2016, ECCV.

[31]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[32]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[33]  Alexei A. Efros,et al.  Generative Visual Manipulation on the Natural Image Manifold , 2016, ECCV.

[34]  R. Murphy Building cell models and simulations from microscope images. , 2016, Methods.

[35]  Yann LeCun,et al.  Disentangling factors of variation in deep representation using adversarial training , 2016, NIPS.

[36]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[37]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[38]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[39]  Jieyue Li,et al.  Automated Learning of Subcellular Variation among Punctate Protein Patterns and a Generative Model of Their Relation to Microtubules , 2015, PLoS Comput. Biol..

[40]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[41]  Alex Graves,et al.  Conditional Image Generation with PixelCNN Decoders , 2016, NIPS.

[42]  Federico Vaggi,et al.  Reconstructing regulatory pathways by systematically mapping protein localization interdependency networks , 2017, bioRxiv.

[43]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[44]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[45]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[46]  Yinda Zhang,et al.  LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop , 2015, ArXiv.

[47]  Pascal Vincent,et al.  Generalized Denoising Auto-Encoders as Generative Models , 2013, NIPS.

[48]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  David Lopez-Paz,et al.  Revisiting Classifier Two-Sample Tests , 2016, ICLR.

[50]  Bernhard Schölkopf,et al.  AdaGAN: Boosting Generative Models , 2017, NIPS.

[51]  Bálint Antal,et al.  The Image Data Resource: A Scalable Platform for Biological Image Data Access, Integration, and Dissemination , 2016, bioRxiv.

[52]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[53]  Euan A. Ashley,et al.  Deep Learning Automates the Quantitative Analysis of Individual Cells in Live-Cell Imaging Experiments , 2016, PLoS Comput. Biol..

[54]  Thomas D. Pollard,et al.  Understanding cytokinesis: lessons from fission yeast , 2010, Nature Reviews Molecular Cell Biology.

[55]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.