Visual reconstruction from 2-photon calcium imaging suggests linear readout properties of neurons in mouse primary visual cortex

Deciphering the neural code, that is interpreting the responses of sensory neurons from the perspective of a downstream population, is an important step towards understanding how the brain processes sensory stimulation. While previous work has focused on classification algorithms to identify the most likely stimulus label in a predefined set of categories, fewer studies have approached a full stimulus reconstruction task. Outstanding questions revolve around the type of algorithm that is most suited to decoding (i.e. full reconstruction, in the context of this study), especially in the presence of strong encoding non-linearities, and the possible role of pairwise correlations. We present, here, the first pixel-by-pixel reconstruction of a complex natural stimulus from 2-photon calcium imaging responses of mouse primary visual cortex (V1). We decoded the activity of approximately 100 neurons from layer 2/3 using an optimal linear estimator and an artificial neural network. We also investigated how much accuracy is lost in this decoding operation when ignoring pairwise neural correlations. We found that a simple linear estimator is sufficient to extract relevant stimulus features from the neural responses, and that it was not significantly outperformed by a non-linear decoding algorithm. The importance of pairwise correlations for reconstruction accuracy was also limited. The results of this study suggest that, conditional on the spatial and temporal limits of the recording technique, V1 neurons display linear readout properties, with low information content in the joint distribution of their activity.