Optimal planning to plan: People partially plan based on plan specificity

Planning requires simulating future choices and consequences. This process is costly. But, it is also useful since it allows people to make choices in the now that have desirable future outcomes. What is a rational way to balance the immediate computational costs and future benefits of planning? Here, we argue that this involves planning to plan—adaptively deciding what actions to plan and when to plan those actions. To formalize this intuition, we develop the ideas of partial planning and information-theoretic simulation costs. Together, these allow us to define a novel Bellman objective that includes both environmental rewards and planning costs, which we solve using a gradient-based planning-to-plan algorithm. A key prediction of our account is that when the value of an immediate action depends on a more specific plan, the computational cost associated with that action will be higher. To test this qualitative prediction, we measure participant response times when solving a Gridworld task. We find evidence for our account of planning costs, indicating that people rationally plan to plan. Our formulation and results provide new insight into the meta-planning processes that support the scale and sophistication of human problem solving.