Dorsal anterior cingulate-midbrain ensemble as a reinforcement meta-learner

The dorsal anterior cingulate cortex (dACC) is central in higher-order cognition and behavioural flexibility. Reinforcement Learning, Bayesian decision-making, and cognitive control are currently the three main theoretical frameworks within which the elusive computational nature of this brain area is chased after–but with overall limited success. Here we propose a new model–the Reinforcement Meta Learner (RML)–in which we exploit core insights into the anatomical connections of the ACC with two midbrain catecholamine nuclei (VTA and LC). With its dual role of selecting and implementing optimal decisions (via VTA) and learning to control its own learning parameters (via LC), the RML generates an autonomous control system with the ability of learning to solve hierarchical decision problems without having an intrinsic hierarchical structure itself. We discuss how our model accounts for an unprecedented number of empirical findings across various cognitive domains, assimilates various previously proposed ACC computations while respecting biological constraints, and provides theoretical integration at various levels. The theoretical pillars of our work promise a generic template (i.e., recurrent connectivity between cortex and midbrain) with which meta-cognition can be computationally approached.