How to Identify Strategy Use and Adaptive Strategy Selection: The Crucial Role of Chance Correction in Weighted Compensatory Strategies

Model comparisons based on choices are a common method to assess adaptive strategy selection models. We show that one common methodological choice prevailing in parts of this work makes it hard to draw sound conclusions concerning the central questions of how often people indeed use simplified strategies (e.g., fast-and-frugal heuristics) and whether they select between them adaptively. Conclusions from this work are biased insofar that more complex weighted compensatory strategies, such as Franklin's Rule, that are considered alternatives to fast-and-frugal heuristics are specified in a way that leads to a systematic underestimation of the degree to which they are relied on. Specifically, the common practice of not correcting cue validities for chance level results in a substantial disadvantage for weighted compensatory strategies. Model recovery simulations and a reanalysis of published data correspondingly show overestimation of the reliance on fast-and-frugal heuristics for various environments used in prominent studies. Furthermore, our results indicate that the magnitude of this overestimation is contingent on the structure of the environment so that strategy selection is mimicked without any material basis. As a result, even completely rational individuals would be erroneously classified as adaptive users of simple strategies. We recommend that chance corrected implementations of weighted compensatory strategies should be included in all future studies investigating the usage of adaptive strategies and argue that comprehensive re-analyses of previous work are warranted. Copyright © 2016 John Wiley & Sons, Ltd.

[1]  E. Brunswik.  Representative design and probabilistic theory in a functional psychology. , 1955, Psychological review.

[2]  K. R. Hammond,et al.  ANALYZING THE COMPONENTS OF CLINICAL INFERENCE. , 1964, Psychological review.

[3]  Michael H. Birnbaum,et al.  The devil rides again: Correlation as an index of fit. , 1973 .

[4]  H. Simon.  Method and Appraisal in Economics: From substantive to procedural rationality , 1976 .

[5]  J. Shanteau.  Correlation as a deceiving measure of fit , 1977 .

[6]  L. Beach,et al.  A Contingency Model for the Selection of Decision Strategies , 1978 .

[7]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[8]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[9]  Eric J. Johnson,et al.  Adaptive Strategy Selection in Decision Making. , 1988 .

[10]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[11]  John W. Payne,et al.  The adaptive decision maker: Name index , 1993 .

[12]  J. Townsend,et al.  Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. , 1993, Psychological review.

[13]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[14]  Gerd Gigerenzer,et al.  Surrogates for Theories , 1998 .

[15]  C. Gettys,et al.  MINERVA-DM: A memory processes model for judgments of likelihood. , 1999 .

[16]  A. Bröder.  Assessing the empirical validity of the "take-the-best" heuristic as a model of human probabilistic inference. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[17]  Wasserman,et al.  Bayesian Model Selection and Model Averaging. , 2000, Journal of mathematical psychology.

[18]  H Pashler,et al.  How persuasive is a good fit? A comment on theory testing. , 2000, Psychological review.

[19]  M. Dhami.  Journal of Behavioral Decision Making J. Behav. Dec. Making, 14: 141±168 �2001) DOI: 10.1002/bdm.371 Bailing and Jailing the Fast and Frugal Way , 2022 .

[20]  Mark A. Pitt,et al.  Toward a method of selecting among computational models of cognition. , 2002 .

[21]  I. J. Myung,et al.  When a good fit can be bad , 2002, Trends in Cognitive Sciences.

[22]  A. Bröder.  Decision making with the "adaptive toolbox": influence of environmental structure, intelligence, and working memory load. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[23]  B. Newell,et al.  Take the best or look at the rest? Factors influencing "one-reason" decision making. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[24]  Mandeep K. Dhami,et al.  Psychological Models of Professional Decision Making , 2003, Psychological science.

[25]  B. Newell,et al.  Empirical tests of a fast-and-frugal heuristic: Not everyone "takes-the-best" , 2003 .

[26]  Arndt Bröder,et al.  Bayesian strategy assessment in multi‐attribute decision making , 2003 .

[27]  A. Bröder,et al.  Take the best versus simultaneous feature matching: probabilistic inferences from memory and effects of representation format. , 2003, Journal of experimental psychology. General.

[28]  David R Shanks,et al.  On the role of recognition in decision making. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[29]  M. Lee,et al.  Evidence accumulation in decision making: Unifying the “take the best” and the “rational” models , 2004, Psychonomic bulletin & review.

[30]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[31]  Jay I. Myung,et al.  Assessing the distinguishability of models and the informativeness of data , 2004, Cognitive Psychology.

[32]  B. Newell.  Re-visions of rationality? , 2005, Trends in Cognitive Sciences.

[33]  Ben R. Newell,et al.  On the binary quality of recognition and the inconsequentiality of further knowledge: two critical tests of the recognition heuristic , 2006 .

[34]  Ulrich Hoffrage,et al.  The Ecological Rationality of Simple Group Heuristics: Effects of Group Member Strategies on Decision Accuracy , 2006 .

[35]  Jörg Rieskamp,et al.  Perspectives of probabilistic inferences: Reinforcement learning and an adaptive network compared. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[36]  A. Bröder,et al.  Adaptive flexibility and maladaptive routines in selecting fast and frugal decision strategies. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[37]  J. Rieskamp,et al.  SSL: a theory of how people learn to select strategies. , 2006, Journal of experimental psychology. General.

[38]  Arndt Bröder,et al.  The use of recognition information and additional cues in inferences from memory. , 2006, Acta psychologica.

[39]  Anders Winman,et al.  The naïve intuitive statistician: a naïve sampling model of intuitive confidence intervals. , 2007, Psychological review.

[40]  Jörg Rieskamp,et al.  The influence of information redundancy on probabilistic inferences , 2007, Memory & cognition.

[41]  L. Schooler,et al.  The aging decision maker: cognitive aging and the adaptive selection of decision strategies. , 2007, Psychology and aging.

[42]  W. Gaissmaier,et al.  Sequential processing of cues in memory-based multiattribute decisions , 2007, Psychonomic bulletin & review.

[43]  R. Hogarth,et al.  Heuristic and linear models of judgment: matching rules and environments. , 2007, Psychological review.

[44]  Ulrich Hoffrage,et al.  When One Cue is not Enough: Combining Fast and Frugal Heuristics with Compound Cue Processing , 2007, Quarterly journal of experimental psychology.

[45]  S. Wichary,et al.  Individual differences in decisiveness: pre-decisional information search and decision strategy use , 2008 .

[46]  A. Glöckner,et al.  Do People Make Decisions Under Risk Based on Ignorance? An Empirical Test of the Priority Heuristic Against Cumulative Prospect Theory , 2008 .

[47]  R. Hogarth,et al.  Determinants of Linear Judgment: A Meta-Analysis of Lens Model Studies , 2007, Psychological bulletin.

[48]  Benjamin E Hilbig,et al.  Recognizing users of the recognition heuristic. , 2008, Experimental psychology.

[49]  M. Birnbaum,et al.  Dimension integration: Testing models without trade-offs , 2008 .

[50]  M. Birnbaum,et al.  New Paradoxes of Risky Decision Making , 2022 .

[51]  J. Rieskamp.  The importance of learning when making inferences , 2008, Judgment and Decision Making.

[52]  Julian N. Marewski,et al.  The recognition heuristic in memory‐based inference: is recognition a non‐compensatory cue? , 2008 .

[53]  A. Glöckner,et al.  Multiple-reason decision making based on automatic processing. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[54]  Andreas Glöckner,et al.  Modeling Option and Strategy Choices with Connectionist Networks: Towards an Integrative Model of Automatic and Deliberate Decision Making , 2008, Judgment and Decision Making.

[55]  B. Hilbig.  One-reason decision making in risky choice? A closer look at the priority heuristic , 2008 .

[56]  Gerd Gigerenzer,et al.  Homo Heuristicus: Why Biased Minds Make Better Inferences , 2009, Top. Cogn. Sci..

[57]  Sara D. Hodges,et al.  Parallel Constraint Satisfaction in Memory-Based Decisions , 2009 .

[58]  Mandeep K. Dhami,et al.  Take-the-best in expert – novice decision strategies for residential burglary , 2022 .

[59]  S. Ayal,et al.  Ignorance or integration: the cognitive processes underlying choice behavior , 2009 .

[60]  Benjamin E. Hilbig,et al.  Ignorance- versus evidence-based decision making: a decision time analysis of the recognition heuristic. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[61]  A. Glöckner,et al.  Coherence Shifts in Probabilistic Inference Tasks , 2009 .

[62]  Jörg Rieskamp,et al.  Do people treat missing information adaptively when making inferences? , 2009, Quarterly journal of experimental psychology.

[63]  A. Glöckner.  Investigating intuitive and deliberate processes statistically: The multiple-measure maximum likelihood strategy classification method , 2009, Judgment and Decision Making.

[64]  J. Rieskamp,et al.  Learning to choose: Cognitive aging and strategy selection learning in decision making. , 2010, Psychology and aging.

[65]  A. Glöckner,et al.  Beyond dual-process models: A categorisation of processes underlying intuitive judgement and decision making , 2010 .

[66]  Lael J. Schooler,et al.  From recognition to decisions: Extending and testing recognition-based models for multialternative inference , 2010, Psychonomic bulletin & review.

[67]  Brent Snook,et al.  Modelling Police Officers' Judgements of the Veracity of Suicide Notes , 2010 .

[68]  Benjamin E. Hilbig,et al.  Reconsidering “evidence” for fast-and-frugal heuristics , 2010, Psychonomic bulletin & review.

[69]  Marc Jekel,et al.  Implementation of the multiple-measure maximum likelihood strategy classification in R , 2010 .

[70]  Andreas Glöckner,et al.  Accounting for critical evidence while being precise and avoiding the strategy selection problem in a parallel constraint satisfaction approach – A reply to Marewski , 2010 .

[71]  David V. Budescu,et al.  Why recognition is rational: Optimality results on single-variable decision rules , 2010, Judgment and Decision Making.

[72]  Julian N. Marewski.  On the theoretical precision and strategy selection problem of a single-strategy approach: A comment on Glockner, Betsch, and Schindler (2010). , 2010 .

[73]  Edgar Erdfelder,et al.  One-reason decision making unveiled: a measurement model of the recognition heuristic. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[74]  Andreas Glöckner,et al.  Physiological arousal in processing recognition information: Ignoring or integrating cognitive cues? , 2010, Judgment and Decision Making.

[75]  Ben R. Newell,et al.  Recognising the recognition heuristic for what it is (and what it's not) , 2011 .

[76]  Andreas Glöckner,et al.  Processing of recognition information and additional cues: A model-based analysis of choice, confidence, and response time , 2011, Judgment and Decision Making.

[77]  Andreas Glöckner,et al.  An eye‐tracking study on information processing in risky decisions: Evidence for compensatory strategies based on automatic processes , 2011 .

[78]  Gerd Gigerenzer,et al.  Heuristic decision making. , 2011, Annual review of psychology.

[79]  Jörg Rieskamp,et al.  When easy comes hard: the development of adaptive strategy selection. , 2011, Child development.

[80]  A. Glöckner,et al.  The empirical content of theories in judgment and decision making: Shortcomings and remedies , 2011, Judgment and Decision Making.

[81]  Mandeep K. Dhami,et al.  Simply Criminal: Predicting Burglars’ Occupancy Decisions with a Simple Heuristic , 2011, Law and human behavior.

[82]  Marc Jekel,et al.  Diagnostic Task Selection for Strategy Classification in Judgment and Decision Making: Theory, Validation, and Implementation in R , 2011, Judgment and Decision Making.

[83]  Andreas Glöckner,et al.  Decisions beyond boundaries: when more information is processed faster than less. , 2012, Acta psychologica.

[84]  Marc Jekel,et al.  The rationality of different kinds of intuitive decision processes , 2012, Synthese.

[85]  Wolfgang Gaissmaier,et al.  The environment matters: Comparing individuals and dyads in their adaptive use of decision strategies , 2013, Judgment and Decision Making.

[86]  Jörg Rieskamp,et al.  Testing adaptive toolbox models: a Bayesian hierarchical approach. , 2013, Psychological review.

[87]  A. Glöckner,et al.  What is adaptive about adaptive decision making? A parallel constraint satisfaction account , 2014, Cognition.

[88]  B. Hilbig,et al.  Generalized outcome-based strategy classification: Comparing deterministic and probabilistic choice models , 2014, Psychonomic Bulletin & Review.

[89]  Brian A. Nosek,et al.  Promoting an open research culture , 2015, Science.

[90]  Jan Rummel,et al.  Effects of ego-depletion on choice behaviour in a multi-attribute decision task , 2016 .

[91]  A. Voss,et al.  Additional information is not ignored: New evidence for information integration and inhibition in take-the-best decisions. , 2016, Acta psychologica.