Mapping mesoscale axonal projections in the mouse brain using a 3D convolutional network

Significance Simple, high-resolution methods for visualizing complex neural circuitry in 3D in the intact mammalian brain are revolutionizing the way researchers study brain connectivity and function. However, concomitant development of robust, open-source computational tools for the automated quantification and analysis of these volumetric data has not kept pace. We have developed a method to perform automated identifications of axonal projections in whole mouse brains. Our method takes advantage of recent advances in machine learning and outperforms existing methods in ease of use, speed, accuracy, and generalizability for axons from different types of neurons. The projection targets of a neuronal population are a key feature of its anatomical characteristics. Historically, tissue sectioning, confocal microscopy, and manual scoring of specific regions of interest have been used to generate coarse summaries of mesoscale projectomes. We present here TrailMap, a three-dimensional (3D) convolutional network for extracting axonal projections from intact cleared mouse brains imaged by light-sheet microscopy. TrailMap allows region-based quantification of total axon content in large and complex 3D structures after registration to a standard reference atlas. The identification of axonal structures as thin as one voxel benefits from data augmentation but also requires a loss function that tolerates errors in annotation. A network trained with volumes of serotonergic axons in all major brain regions can be generalized to map and quantify axons from thalamocortical, deep cerebellar, and cortical projection neurons, validating transfer learning as a tool to adapt the model to novel categories of axonal morphology. Speed of training, ease of use, and accuracy improve over existing tools without a need for specialized computing hardware. Given the recent emphasis on genetically and functionally defining cell types in neural circuit analysis, TrailMap will facilitate automated extraction and quantification of axons from these specific cell types at the scale of the entire mouse brain, an essential component of deciphering their connectivity.