Learning optimal decisions with confidence

Diffusion decision models are immensely successful models for human and animal decisions under uncertainty. The decisions they model require the temporal accumulation of evidence to improve choice accuracy, and thus balance the trade-off between the decisions’ speed and their accuracy. Commonly, diffusion models have a one-dimensional abstract input that represents noisy momentary decision-related evidence. However, the nervous system typically uses population codes to represent sensory variables, which implies that the momentary evidence is distributed across multiple inputs. It is currently unknown how decision makers could learn how to combine these multiple inputs to obtain a one-dimensional abstract decision variable. We present here a Bayesian learning rule for learning a near-optimal linear combination of these inputs based on trial-by-trial feedback. The rule is Bayesian in the sense that it learns not only the mean of the weights but also the uncertainty around this mean in the form of a covariance matrix. This yields a rule whose learning rate is strongly modulated by decision confidence, providing a computational role of the latter for every-day decisions. Furthermore, we show that, in volatile environments, the rule predicts a bias towards repeating the same choice after correct decisions, with a bias strength that is modulated by the previous choice’ difficulty. Last, we extend our learning rule to cases for which one of the choices is more likely a-priori, which provides new insights into how such biases modulate the mechanisms leading to optimal decisions in diffusion models.