Multi-scale generative adversarial network for improved evaluation of cell–cell interactions observed in organ-on-chip experiments

Organs On a Chip (OOCs) represent a sophisticated approach for exploring biological mechanisms and developing therapeutic agents. In conjunction with high-quality time-lapse microscopy (TLM), OOCs allow for the visualization of reconstituted complex biological processes, such as multi-cell-type migration and cell–cell interactions. In this context, increasing the frame rate is desirable to reconstruct accurately cell-interaction dynamics. However, a trade-off between high resolution and carried information content is required to reduce the overall data volume. Moreover, high frame rates increase photobleaching and phototoxicity. As a possible solution for these problems, we report a new hybrid-imaging paradigm based on the integration of OOC/TLMs with a Multi-scale Generative Adversarial Network (GAN) predicting interleaved video frames with the aim to provide high-throughput videos. We tested the performance of the predictive capability of GAN on synthetic videos, as well as on real OOC experiments dealing with tumor–immune cell interactions. The proposed approach offers the possibility to acquire a reduced number of high-quality TLM images without any major loss of information on the phenomena under investigation.

[1]  J. Munkres.  ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[2]  Tao Xu,et al.  SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation , 2017, Neuroinformatics.

[3]  Joost B. Beltman,et al.  Analysing immune cell migration , 2009, Nature Reviews Immunology.

[4]  Eric P. Xing,et al.  Dual Motion GAN for Future-Flow Embedded Video Prediction , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Sara Reardon,et al.  ‘Organs-on-chips’ go mainstream , 2015, Nature.

[6]  Sarah E Henrickson,et al.  Towards estimating the true duration of dendritic cell interactions with T cells. , 2009, Journal of immunological methods.

[7]  Patrice Dosset,et al.  Automatic detection of diffusion modes within biological membranes using back-propagation neural network , 2016, BMC Bioinformatics.

[8]  S. van de Linde,et al.  Light-induced cell damage in live-cell super-resolution microscopy , 2015, Scientific Reports.

[9]  Javier Selva Castelló.  A Comprehensive survey on deep future frame video prediction , 2018 .

[10]  Antonio Torralba,et al.  Generating Videos with Scene Dynamics , 2016, NIPS.

[11]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Haw Yang,et al.  Quantitative characterization of changes in dynamical behavior for single-particle tracking studies. , 2006, Journal of Physical Chemistry B.

[13]  Kenneth D. Harris,et al.  Community-based benchmarking improves spike rate inference from two-photon calcium imaging data , 2018, PLoS Comput. Biol..

[14]  Subaji Mohan,et al.  ROI-based feature learning for efficient true positive prediction using convolutional neural network for lung cancer diagnosis , 2020, Neural Computing and Applications.

[15]  Ruth E. Baker,et al.  The impact of temporal sampling resolution on parameter inference for biological transport models , 2017, PLoS Comput. Biol..

[16]  Jianwu Li,et al.  Removing ring artifacts in CBCT images via generative adversarial networks with unidirectional relative total variation loss , 2019, Neural Computing and Applications.

[17]  Rafael C. González,et al.  Digital image processing using MATLAB , 2006 .

[18]  Shuai Li,et al.  Online Optimization Methods for the Quantification Problem , 2016, KDD.

[19]  Elena Agliari,et al.  Cancer-driven dynamics of immune cells in a microfluidic environment , 2014, Scientific Reports.

[20]  Shuai Li,et al.  Improved Algorithm on Online Clustering of Bandits , 2019, IJCAI.

[21]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[22]  A. Ozcan,et al.  Deep learning enables cross-modality super-resolution in fluorescence microscopy , 2018, Nature Methods.

[23]  Shuai Li.  The art of clustering bandits , 2016 .

[24]  D. Beebe,et al.  The present and future role of microfluidics in biomedical research , 2014, Nature.

[25]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[26]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Jürgen Köhler,et al.  Probing the type of anomalous diffusion with single-particle tracking. , 2014, Physical chemistry chemical physics : PCCP.

[29]  Xiaoou Tang,et al.  Video Frame Synthesis Using Deep Voxel Flow , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[31]  E. R. Davies,et al.  Machine vision - theory, algorithms, practicalities , 2004 .

[32]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[33]  Cordelia Schmid,et al.  EpicFlow: Edge-preserving interpolation of correspondences for optical flow , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Mohammad Mansouri,et al.  An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets , 2018, Nature Biomedical Engineering.

[35]  C Di Natale,et al.  The influence of spatial and temporal resolutions on the analysis of cell-cell interaction: a systematic study for time-lapse microscopy applications , 2019, Scientific Reports.

[36]  Arianna Mencattini,et al.  3D Microfluidic model for evaluating immunotherapy efficacy by tracking dendritic cell behaviour toward tumor cells , 2017, Scientific Reports.

[37]  Jan Kautz,et al.  MoCoGAN: Decomposing Motion and Content for Video Generation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  V. Baumans,et al.  Use of animals in experimental research: an ethical dilemma? , 2004, Gene therapy.

[39]  Cordelia Schmid,et al.  DeepFlow: Large Displacement Optical Flow with Deep Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[40]  Raul H. C. Lopes,et al.  Pengaruh Latihan Small Sided Games 4 Lawan 4 Dengan Maksimal Tiga Sentuhan Terhadap Peningkatan VO2MAX Pada Siswa SSB Tunas Muda Bragang Klampis U-15 , 2022, Jurnal Ilmiah Mandala Education.

[41]  Annamaria Gerardino,et al.  Cross talk between cancer and immune cells: exploring complex dynamics in a microfluidic environment. , 2013, Lab on a chip.

[42]  Morteza Mardani,et al.  Deep Generative Adversarial Neural Networks for Compressive Sensing MRI , 2019, IEEE Transactions on Medical Imaging.

[43]  A. Barra,et al.  Organs on chip approach: a tool to evaluate cancer -immune cells interactions , 2017, Scientific Reports.

[44]  Federico Vaggi,et al.  GANs for Biological Image Synthesis , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  Sergey Levine,et al.  Stochastic Adversarial Video Prediction , 2018, ArXiv.

[46]  Arianna Mencattini,et al.  Dissecting Effects of Anti-cancer Drugs and Cancer-Associated Fibroblasts by On-Chip Reconstitution of Immunocompetent Tumor Microenvironments. , 2018, Cell reports.

[47]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[48]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[49]  Dinggang Shen,et al.  Medical Image Synthesis with Deep Convolutional Adversarial Networks , 2018, IEEE Transactions on Biomedical Engineering.

[50]  James Zou,et al.  Feedback GAN for DNA optimizes protein functions , 2019, Nature Machine Intelligence.

[51]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[52]  Shuai Li,et al.  Distributed Clustering of Linear Bandits in Peer to Peer Networks , 2016, ICML.

[53]  Shuai Li,et al.  On Context-Dependent Clustering of Bandits , 2016, ICML.

[54]  L. Zitvogel,et al.  Chemotherapy-induced antitumor immunity requires formyl peptide receptor 1 , 2015, Science.

[55]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.