Neuronal Learning Analysis using Cycle-Consistent Adversarial Networks

Understanding how activity in neural circuits reshapes following task learning could reveal fundamental mechanisms of learning. Thanks to the recent advances in neural imaging technologies, high-quality recordings can be obtained from hundreds of neurons over multiple days or even weeks. However, the complexity and dimensionality of population responses pose significant challenges for analysis. Existing methods of studying neuronal adaptation and learning often impose strong assumptions on the data or model, resulting in biased descriptions that do not generalize. In this work, we use a variant of deep generative models called – cycle-consistent adversarial networks, to learn the unknown mapping between preand post-learning neuronal activities recorded in vivo. To do so, we develop an end-to-end pipeline to preprocess, train and evaluate calcium fluorescence signals, and a procedure to interpret the resulting deep learning models. To assess the validity of our method, we first test our framework on a synthetic dataset with known ground-truth transformation. Subsequently, we applied our method to neuronal activities recorded from the primary visual cortex of behaving mice, where the mice transition from novice to expert-level performance in a visual-based virtual reality experiment. We evaluate model performance on generated calcium imaging signals and their inferred spike trains. To maximize performance, we derive a novel approach to pre-sort neurons such that convolutional-based networks can take advantage of the spatial information that exists in neuronal activities. In addition, we incorporate visual explanation methods to improve the interpretability of our work and gain insights into the learning process as manifested in the cellular activities. Together, our results demonstrate that analyzing neuronal learning processes with data-driven deep unsupervised methods holds the potential to unravel changes in an unbiased way.

[1]  Omid G. Sani,et al.  Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification , 2020, Nature Neuroscience.

[2]  Matthias Bethge,et al.  Benchmarking Spike Rate Inference in Population Calcium Imaging , 2016, Neuron.

[3]  Lukasz Kaiser,et al.  Unsupervised Cipher Cracking Using Discrete GANs , 2018, ICLR.

[4]  Stefano Panzeri,et al.  Synthesizing realistic neural population activity patterns using Generative Adversarial Networks , 2018, ICLR.

[5]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[6]  Chethan Pandarinath,et al.  Inferring single-trial neural population dynamics using sequential auto-encoders , 2017, Nature Methods.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Lena Constantin,et al.  Calcium imaging and the curse of negativity , 2020 .

[9]  Krzysztof Rataj,et al.  Mol-CycleGAN: a generative model for molecular optimization , 2019, Journal of Cheminformatics.

[10]  Arno Onken,et al.  CalciumGAN: A Generative Adversarial Network Model for Synthesising Realistic Calcium Imaging Data of Neuronal Populations , 2020, ArXiv.

[11]  Philip Bachman,et al.  Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data , 2018, ICML.

[12]  Mark C. W. van Rossum,et al.  A Novel Spike Distance , 2001, Neural Computation.

[13]  Fritjof Helmchen,et al.  A database and deep learning toolbox for noise-optimized, generalized spike inference from calcium imaging , 2020, Nature Neuroscience.

[14]  Giancarlo Fortino,et al.  A survey on deep learning in medicine: Why, how and when? , 2021, Inf. Fusion.

[15]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[16]  Lovedeep Gondara,et al.  Medical Image Denoising Using Convolutional Denoising Autoencoders , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).

[17]  Jakob H. Macke,et al.  Adversarial Training of Neural Encoding Models on Population Spike Trains , 2019, 2019 Conference on Cognitive Computational Neuroscience.

[18]  Hongxun Yao,et al.  Auto-encoder based dimensionality reduction , 2016, Neurocomputing.

[19]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[20]  Nathalie L. Rochefort,et al.  The Impact of Visual Cues, Reward, and Motor Feedback on the Representation of Behaviorally Relevant Spatial Locations in Primary Visual Cortex , 2018, Cell reports.

[21]  Kenneth D. Harris,et al.  Neuropixels 2.0: A miniaturized high-density probe for stable, long-term brain recordings , 2020, Science.

[22]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Surya Ganguli,et al.  Unsupervised Discovery of Demixed, Low-Dimensional Neural Dynamics across Multiple Timescales through Tensor Component Analysis , 2017, Neuron.

[24]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[25]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[26]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[27]  Noureddine Zerhouni,et al.  Deep Learning in the Biomedical Applications: Recent and Future Status , 2019, Applied Sciences.

[28]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Andre Araujo,et al.  Computing Receptive Fields of Convolutional Neural Networks , 2019, Distill.

[31]  Jonathan Tompson,et al.  Efficient object localization using Convolutional Networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[33]  Virginia Rutten,et al.  Non-reversible Gaussian processes for identifying latent dynamical structure in neural data , 2020, NeurIPS.

[34]  John P. Cunningham,et al.  Linear dynamical neural population models through nonlinear embeddings , 2016, NIPS.

[35]  Feng Liu,et al.  Deep Learning and Its Applications in Biomedicine , 2018, Genom. Proteom. Bioinform..

[36]  CALFADS: LATENT FACTOR ANALYSIS OF DYNAMI- CAL SYSTEMS IN CALCIUM IMAGING DATA , 2020 .

[37]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[38]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[39]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Zewen Li,et al.  A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[41]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[42]  Nathalie L Rochefort,et al.  Reward Association Enhances Stimulus-Specific Representations in Primary Visual Cortex , 2020, Current Biology.

[43]  Jacob Abernethy,et al.  On Convergence and Stability of GANs , 2018 .

[44]  Olivier Bachem,et al.  Recent Advances in Autoencoder-Based Representation Learning , 2018, ArXiv.