Neural Population Control via Deep ANN Image Synthesis

Specific deep feed-forward artificial neural networks (ANNs) constitute our current best understanding of the primate ventral visual stream and the core object recognition behavior it supports. Here we turn things around and ask: can we use these ANN models to synthesize images that control neural activity? We here test this control in cortical area V4 in two control goal settings. i) Single Neuron State Control: ”stretch” the maximal firing rate of any single neural site beyond its naturally occurring maximal rate. ii) Population State Control: independently and simultaneously control all neural sites in a population. Specifically, we aimed to push the recorded population into a ”one hot” state in which one neural site is active and all others are clamped at baseline. We report that, using ANN-driven image synthesis, we can drive the firing rates of most V4 neural sites beyond naturally occurring levels. And we report that V4 neural sites with overlapping receptive fields can be partly – but not yet perfectly – independently controlled. These results are the strongest test thus far of deep ANN models of the ventral stream, and they show how these models could be used for setting desired brain states.