Structural properties of individual instances predict human effort and performance on an NP-Hard problem

Life presents us with decisions of varying degrees of difficulty. Many of them are NP-hard, that is, they are computationally intractable. Two important questions arise: which properties of decisions drive extreme computational hardness and what are the effects of these properties on human-decision making? Here, we postulate that we can study the effects of computational complexity on human decision-making by studying the mathematical properties of individual instances of NP-hard problems. We draw on prior work in computational complexity theory, which suggests that computational difficulty can be characterized based on the features of instances of a problem. This study is the first to apply this approach to human decision-making. We measured hardness, first, based on typical-case complexity (TCC), a measure of average complexity of a random ensemble of instances, and, second, based on instance complexity (IC), a measure that captures the hardness of a single instance of a problem, regardless of the ensemble it came from. We tested the relation between these measures and (i) decision quality as well as (ii) time expended in a decision, using two variants of the 0-1 knapsack problem, a canonical and ubiquitous computational problem. We show that participants expended more time on instances with higher complexity but that decision quality was lower in those instances. These results suggest that computational complexity is an inherent property of the instances of a problem, which affect human and other kinds of computers.

[1]  Giorgio Gambosi,et al.  Complexity and approximation: combinatorial optimization problems and their approximability properties , 1999 .

[2]  Sanjeev Arora,et al.  Computational Complexity: A Modern Approach , 2009 .

[3]  Rakefet Ackerman,et al.  Meta-Reasoning: Monitoring and Control of Thinking and Reasoning , 2017, Trends in Cognitive Sciences.

[4]  Peter Bossaerts,et al.  How Humans Solve Complex Problems: The Case of the Knapsack Problem , 2016, Scientific Reports.

[5]  Marie-Pascale Noël,et al.  The detrimental effect of interference in multiplication facts storing: typical development and individual differences. , 2014, Journal of experimental psychology. General.

[6]  Federico Ricci-Tersenghi,et al.  Being Glassy Without Being Hard to Solve , 2010, Science.

[7]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[8]  T. Griffiths,et al.  Strategy Selection as Rational Metareasoning , 2017, Psychological review.

[9]  Maria Otworowska,et al.  Naturalism, tractability and the adaptive toolbox , 2019, Synthese.

[10]  Christopher Y. Olivola,et al.  The Effort Paradox: Effort Is Both Costly and Valued , 2018, Trends in Cognitive Sciences.

[11]  Niklas Sörensson,et al.  An Extensible SAT-solver , 2003, SAT.

[12]  Alice Y. Chiang,et al.  Working-memory capacity protects model-based learning from stress , 2013, Proceedings of the National Academy of Sciences.

[13]  Scott Aaronson,et al.  Guest Column: NP-complete problems and physical reality , 2005, SIGA.

[14]  Mark H. Ashcraft,et al.  A network approach to mental multiplication. , 1982 .

[15]  Marinella Cappelletti,et al.  Spared numerical abilities in a case of semantic dementia , 2001, Neuropsychologia.

[16]  Assaf Naor,et al.  Rigorous location of phase transitions in hard optimization problems , 2005, Nature.

[17]  B. Schmeichel,et al.  Attention control, memory updating, and emotion regulation temporarily reduce the capacity for executive control. , 2007, Journal of experimental psychology. General.

[18]  Rémi Monasson,et al.  Determining computational complexity from characteristic ‘phase transitions’ , 1999, Nature.

[19]  Guilhem Semerjian,et al.  Biased landscapes for random constraint satisfaction problems , 2018, Journal of Statistical Mechanics: Theory and Experiment.

[20]  Thomas L. Griffiths,et al.  Algorithm selection by rational metareasoning as a model of human strategy selection , 2014, NIPS.

[21]  Peter Bossaerts,et al.  Computational Complexity and Human Decision-Making , 2017, Trends in Cognitive Sciences.

[22]  Johan Kwisthout,et al.  Cognition and intractability: a guide to classical and parameterized complexity analysis , 2019 .

[23]  Ed Blakey.  Computational Complexity in Non-Turing Models of Computation: The What, the Why and the How , 2011, Electron. Notes Theor. Comput. Sci..

[24]  Lenka Zdeborová,et al.  Constraint satisfaction problems with isolated solutions are hard , 2008, ArXiv.

[25]  Thomas L. Griffiths,et al.  Rational metareasoning and the plasticity of cognitive control , 2018, PLoS Comput. Biol..

[26]  Sebastian Sardiña,et al.  Phase transition in the knapsack problem , 2018, ArXiv.

[27]  Santosh S. Vempala,et al.  The complexity of human computation via a concrete model with an application to passwords , 2020, Proceedings of the National Academy of Sciences.

[28]  Amin Coja-Oghlan,et al.  On the solution‐space geometry of random constraint satisfaction problems , 2011, Random Struct. Algorithms.

[29]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[30]  Ivan Bratko,et al.  Search-Based Estimation of Problem Difficulty for Humans , 2013, AIED.

[31]  Bart Selman,et al.  Critical Behavior in the Computational Cost of Satisfiability Testing , 1996, Artif. Intell..

[32]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[33]  Hong-Wei Xue,et al.  Arabidopsis PROTEASOME REGULATOR1 is required for auxin-mediated suppression of proteasome activity and regulates auxin signalling , 2016, Nature Communications.

[34]  Douglas Vickers,et al.  Human Performance on Visually Presented Traveling Salesperson Problems with Varying Numbers of Nodes , 2006, J. Probl. Solving.

[35]  Peter Bossaerts,et al.  Promoting Intellectual Discovery: Patents Versus Markets , 2009, Science.

[36]  F. Manes,et al.  “Ecological” and Highly Demanding Executive Tasks Detect Real-Life Deficits in High-Functioning Adult ADHD Patients , 2013, Journal of attention disorders.

[37]  John K. Tsotsos.  Analyzing vision at the complexity level , 1990, Behavioral and Brain Sciences.

[38]  L. E. Arnold,et al.  Helping parents help their children , 1979 .

[39]  Marcello Frixione,et al.  Tractable Competence , 2001, Minds and Machines.

[40]  Falk Lieder,et al.  Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources , 2019, Behavioral and Brain Sciences.

[41]  Christopher Cherniak,et al.  Computational Complexity and the Universal Acceptance of Logic , 1984 .

[42]  R. Selten,et al.  Bounded rationality: The adaptive toolbox , 2000 .

[43]  Guilhem Semerjian,et al.  Typology of phase transitions in Bayesian inference problems , 2018, Physical review. E.

[44]  Toby Walsh,et al.  The TSP Phase Transition , 1996, Artif. Intell..