Large nuisance modulation has little impact on IT target match performance

Many everyday tasks require us to extract a specific type of information from our environment while ignoring other things. When the neurons in our brains that carry task-relevant signals are also modulated by task-irrelevant, ‘nuisance’ information, nuisance modulation is expected to act as performance-limiting noise. To investigate the impact of nuisance modulation on neural task performance, we recorded responses in inferotemporal cortex (IT) as monkeys performed a task in which they were rewarded for indicating when a target object appeared amid considerable nuisance variation. Within IT, we found a robust, behaviorally-relevant target match signal that was mixed with large nuisance modulations in individual neurons. Unexpectedly, we also found that these nuisance modulations had little impact on performance, either within individual IT neurons or across the IT population. We demonstrate how these results follow from fast processing in IT, which placed IT in a low spike count regime where the impact of nuisance variability was blunted by Poisson-like trial variability. These results demonstrate that some basic intuitions about neural coding are misguided in the context of a fast-processing, low spike count regime.