Single-Trial Inhibition of Anterior Cingulate Disrupts Model-based Reinforcement Learning in a Two-step Decision Task.

The anterior cingulate cortex (ACC) is implicated in learning the value of actions, and thus in allowing past outcomes to influence the current choice. However, it is not clear whether or how it contributes to the two major ways such learning is thought to happen: model-based mechanisms that learn action-state predictions and use these to infer action values; or model-free mechanisms which learn action values directly through reward prediction errors. Having confirmed, using a classical probabilistic reversal learning task, that optogenetic inhibition of ACC neurons on single trials indeed affected reinforcement learning, we examined the consequence of this manipulation in a novel two-step decision task designed to dissociate model-free and model-based learning mechanisms in mice. On the two-step task, silencing spared the influence of the trial outcome but reduced the influence of the experienced state transition. Analysis using reinforcement learning models indicated that ACC inhibition disrupted model-based RL mechanisms.