Generic properties of a computational task predict human effort and performance

Abstract It has been shown that computational hardness of cognitive tasks affects people’s effort and ability to solve problems reliably. However, prior empirical studies lack generality. They quantify computational hardness of tasks based on particular algorithms or for specific problems. Here, we propose a set of measures of computational hardness of individual instances of a task in a way that is independent of any algorithm or computational model and can be generalized to other problems. Specifically, we introduce two measures, typical-case complexity (TCC), a measure of average hardness of a random ensemble of instances, and instance complexity (IC), an instance-specific metric. Both measures are related to structural properties of instances. We then test the effect of those measures on human behavior by asking participants to solve instances of two variants of the 0-1 knapsack problem, a canonical and ubiquitous NP-hard problem. We find that participants spent more time on instances with higher TCC and IC, but that decision quality was lower in those instances. We propose that the study of mathematical properties of tasks related to computational hardness can contribute to the development of computationally plausible accounts of human decision-making, just like stochastic properties have proven to be critical to our understanding of human decisions in probabilistic tasks.