Fast and robust active neuron segmentation in two-photon calcium imaging using spatiotemporal deep learning

Significance Two-photon calcium imaging is a standard technique of neuroscience laboratories that records neural activity from individual neurons over large populations in awake-behaving animals. Automatic and accurate identification of behaviorally relevant neurons from these recordings is a critical step toward complete mapping of brain activity. To this end, we present a fast deep learning framework which significantly outperforms previous methods and is the first to be as accurate as human experts in segmenting active and overlapping neurons. Such neuron detection performance is crucial for precise quantification of population-level and single-cell–level neural coding statistics, which will aid neuroscientists to temporally synchronize dynamic behavioral or neural stimulus to the subjects’ neural activity, opening the door for unprecedented accelerated progress in neuroscientific experiments. Calcium imaging records large-scale neuronal activity with cellular resolution in vivo. Automated, fast, and reliable active neuron segmentation is a critical step in the analysis workflow of utilizing neuronal signals in real-time behavioral studies for discovery of neuronal coding properties. Here, to exploit the full spatiotemporal information in two-photon calcium imaging movies, we propose a 3D convolutional neural network to identify and segment active neurons. By utilizing a variety of two-photon microscopy datasets, we show that our method outperforms state-of-the-art techniques and is on a par with manual segmentation. Furthermore, we demonstrate that the network trained on data recorded at a specific cortical layer can be used to accurately segment active neurons from another layer with different neuron density. Finally, our work documents significant tabulation flaws in one of the most cited and active online scientific challenges in neuron segmentation. As our computationally fast method is an invaluable tool for a large spectrum of real-time optogenetic experiments, we have made our open-source software and carefully annotated dataset freely available online.