A Model of Basal Ganglia Function Unifying Reinforcement Learning and Action Selection

We propose a systems-level computational model of the basal gangha based closely on known anatomy and physiology. First, we assume that thalamic output &argets of the basal ganglia, which relay ascending information to cortical action and planning areas, are tonically inhibited. Second, we assume that the output stage of the basal ganglia, the internal segment of the globus pallidus (GPi), selects a winner from several potential actions. The por"entia.1 actions are represented as parallel streams of information, each competing for access to the cortical areas that implement them. The requirement for both tonic inhibition of thalamic nuclei and winnerselection leads to a circuit, that in the simplest possible form, has neurons in exactly the configuration found in the basal ganglia. In particular, the subthalamic nucleus, which contains primarily excitatory neurons, is instrumental in implementing a "winner-lose-all" in the globus pallidus, which selects thalamic targets by disinhibition. We combine this winner-selection mechanism with reinforcement learning through dopaminergic neurons in the substantia nigra and the ventral tegmental area to m o w the cortico-striatal synapse efficacy. Using this model, we demonstrate its function on two behavioral tasks. One task, termed the Multiarmed Bandit (MAB) is based on hypothesis testing in an environment with uncertain risks and rewards. The model both mimics the behavior of normal humans performing this task and makes predictions regarding the performance of schizophrenics. It was found to be sensitive to both the level of presynaptic noise in the striatum and the relative weighting of short-term vs. long-term reward. The performance on the second task, the Wisconsin Card Sorting Test (WCST), was found to be sensitive both to the level of presynaptic noise in the striatum, and the ratio of positive to negative learning rates. The model predicts performance both in Parkinson's disease and schizophrenia. These results are discussed within the framework of existing data regarding mechanisms of synaptic plastiaty, axo-dendritic architectures in the basal ganglia, and disease states.