Re-aligning models of habitual and goal-directed decision-making

Abstract The classic dichotomy between habitual and goal-directed behavior is often mapped onto a dichotomy between model-free and model-based reinforcement learning (RL) algorithms, putatively implemented in segregated neuronal circuits. Despite significant heuristic value in motivating experimental investigations, several lines of evidence suggest that this mapping is in need of modification and/or realignment. First, whereas habitual and goal-directed behaviors have been shown to depend on cleanly separable neural circuitry, recent data suggest that model-based and model-free representations in the brain are largely overlapping. Second, habitual behaviors need not involve representations of expected reinforcement (i.e., need not involve RL, model-free, or otherwise) but may be based instead on simple stimulus–response associations. Finally, goal-directed decisions may not reflect a single model-based algorithm but rather a continuum of “model-basedness.” These lines of evidence thus suggest a possible reconceptualization of the distinction between model-free versus model-based RL—one in which both contribute to a single goal-directed system that is value-based, as opposed to distinct, habitual mechanisms that are value-free. In this chapter, we discuss new models that have extended the RL approach to modeling habitual and goal-directed behavior and assess how these have clarified our understanding of the underlying neural circuitry.