A Comparison of Reinforcement Learning Models for the Iowa Gambling Task Using Parameter Space Partitioning

The Iowa gambling task (IGT) is one of the most popular tasks used to study decisionmaking deficits in clinical populations. In order to decompose performance on the IGT in its constituent psychological processes, several cognitive models have been proposed (e.g., the Expectancy Valence (EV) and Prospect Valence Learning (PVL) models). Here we present a comparison of three models—the EV and PVL models, and a combination of these models (EV-PU)—based on the method of parameter space partitioning. This method allows us to assess the choice patterns predicted by the models across their entire parameter space. Our results show that the EV model is unable to account for a frequency-of-losses effect, whereas the PVL and EV-PU models are unable to account for a pronounced preference for the bad decks with many switches. All three models underrepresent pronounced choice patterns that are frequently seen in experiments. Overall, our results suggest that the search of an appropriate IGT model has not yet come to an end.

[1]  Mark A. Pitt,et al.  Measuring Model Flexibility With Parameter Space Partitioning: An Introduction and Application Example , 2008, Cogn. Sci..

[2]  E. Yechiam,et al.  Non-specific effects of methylphenidate (Ritalin) on cognitive ability and decision-making of ADHD and healthy adults , 2010, Psychopharmacology.

[3]  Benedicto Crespo-Facorro,et al.  Prefrontal cognitive functions in stabilized first-episode patients with schizophrenia spectrum disorders: A dissociation between dorsolateral and orbitofrontal functioning , 2005, Schizophrenia Research.

[4]  Eldad Yechiam,et al.  Comparison of basic assumptions embedded in learning models for experience-based decision making , 2005, Psychonomic bulletin & review.

[5]  Terry E. Goldberg,et al.  Operant conditioning and the orbitofrontal cortex in schizophrenic patients: unexpected evidence for intact functioning , 1998, Schizophrenia Research.

[6]  Jane S. Paulsen,et al.  Neurocognitive deficits related to poor decision making in people behind bars , 2008, Psychonomic bulletin & review.

[7]  Merrill Hiscock,et al.  The Simulated Gambling Paradigm Applied to Young Adults: An Examination of University Students' Performance , 2006, Applied neuropsychology.

[8]  Jerome R. Busemeyer,et al.  Comparison of Decision Learning Models Using the Generalization Criterion Method , 2008, Cogn. Sci..

[9]  R. van den Bos,et al.  Trait anxiety affects decision-making differently in healthy men and women: Towards gender-specific endophenotypes of anxiety , 2010, Neuropsychologia.

[10]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[11]  A. Damasio,et al.  Deciding Advantageously Before Knowing the Advantageous Strategy , 1997, Science.

[12]  Woojae Kim,et al.  A Model-Based fMRI Analysis with Hierarchical Bayesian Parameter Estimation. , 2013, Journal of neuroscience, psychology, and economics.

[13]  Eldad Yechiam,et al.  Working memory and decision-making biases in young adults with a family history of alcoholism: studies from the Oklahoma family health patterns project. , 2006, Alcoholism, clinical and experimental research.

[14]  D. Martino,et al.  Neuropsychological frontal impairments and negative symptoms in schizophrenia , 2007, Psychiatry Research.

[15]  Fintan J. Costello,et al.  Computational modelling of switching behaviour in repeated gambles , 2007, Artificial Intelligence Review.

[16]  Jay I. Myung,et al.  Global model analysis by parameter space partitioning. , 2019, Psychological review.

[17]  A. Malhotra,et al.  Emotion-based decision-making in healthy subjects: short-term effects of reducing dopamine levels , 2006, Psychopharmacology.

[18]  Eldad Yechiam,et al.  Decision-making impairments in adolescents with early-onset schizophrenia , 2006, Schizophrenia Research.

[19]  W. Brown,et al.  Decision-making in individuals with agenesis of the corpus callosum: expectancy-valence in the Iowa Gambling Task. , 2012, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[20]  A. Lawrence,et al.  The somatic marker hypothesis: A critical evaluation , 2006, Neuroscience & Biobehavioral Reviews.

[21]  Eldad Yechiam,et al.  Evaluating the reliance on past choices in adaptive learning models , 2007 .

[22]  David Nussbaum,et al.  Decision Making and Aggression in Forensic Psychiatric Inpatients , 2010 .

[23]  J. Hsieh,et al.  Is deck B a disadvantageous deck in the Iowa Gambling Task? , 2007, Behavioral and Brain Functions.

[24]  R. Blair,et al.  Somatic Markers and Response Reversal: Is There Orbitofrontal Cortex Dysfunction in Boys with Psychopathic Tendencies? , 2001, Journal of abnormal child psychology.

[25]  Yao-Chu Chiu,et al.  Is deck C an advantageous deck in the Iowa Gambling Task? , 2007, Behavioral and Brain Functions.

[26]  Roberto Keller,et al.  Frontal lobe dysfunction in pathological gambling patients , 2002, Biological Psychiatry.

[27]  J. Busemeyer,et al.  A contribution of cognitive decision models to clinical assessment: decomposing performance on the Bechara gambling task. , 2002, Psychological assessment.

[28]  Shlomo Bekhor,et al.  Association of risk proneness in overtaking maneuvers with impaired decision making , 2008 .

[29]  Edwin Cuppen,et al.  The effect of COMT Val158 Met genotype on decision-making and preliminary findings on its interaction with the 5-HTTLPR in healthy females , 2009, Neuropharmacology.

[30]  Matteo Cella,et al.  Cognitive decision modelling of emotion-based learning impairment in schizophrenia: The role of awareness , 2012, Psychiatry Research.

[31]  Jerome R Busemeyer,et al.  Cognitive modeling analysis of decision-making processes in cocaine abusers , 2004, Psychonomic bulletin & review.

[32]  M. Franceschi,et al.  Age-Related Quantitative and Qualitative Changes in Decision Making Ability , 2008, Behavioural neurology.

[33]  Laura Bellodi,et al.  Decision-making heterogeneity in obsessive-compulsive disorder: ventromedial prefrontal cortex function predicts different treatment outcomes , 2002, Neuropsychologia.

[34]  Richard J. Tunney,et al.  Some decks are better than others: The effect of reinforcer type and task instructions on learning in the Iowa Gambling Task , 2006, Brain and Cognition.

[35]  Woojae Kim,et al.  Cognitive Mechanisms Underlying Risky Decision-Making in Chronic Cannabis Users. , 2010, Journal of mathematical psychology.

[36]  Eldad Yechiam,et al.  Recency gets larger as lesions move from anterior to posterior locations within the ventromedial prefrontal cortex , 2010, Behavioural Brain Research.

[37]  P. Brambilla,et al.  Increased salience of gains versus decreased associative learning differentiate bipolar disorder from schizophrenia during incentive decision making , 2012, Psychological Medicine.

[38]  O. Turnbull,et al.  Artificial time constraints on the Iowa Gambling Task: The effects on behavioural performance and subjective experience , 2005, Brain and Cognition.

[39]  Francisco Rubio,et al.  Decision-making impairment on the Iowa Gambling Task after endovascular coiling or neurosurgical clipping for ruptured anterior communicating artery aneurysm. , 2012, Neuropsychology.

[40]  Rosemary Tannock,et al.  Executive and motivational processes in adolescents with Attention-Deficit-Hyperactivity Disorder (ADHD) , 2005, Behavioral and Brain Functions.

[41]  M. Gullo,et al.  Anticipatory Stress Restores Decision-Making Deficits in Heavy Drinkers by Increasing Sensitivity to Losses , 2011, Drug and alcohol dependence.

[42]  A. Simmons,et al.  Emotional decision-making and its dissociable components in schizophrenia and schizoaffective disorder: A behavioural and MRI investigation , 2008, Neuropsychologia.

[43]  Eldad Yechiam,et al.  Decision making in bipolar disorder: A cognitive modeling approach , 2008, Psychiatry Research.

[44]  Wolf Vanpaemel,et al.  Measuring model complexity with the prior predictive , 2009, NIPS.

[45]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[46]  John A. Parkinson,et al.  Emotion‐Based Learning is Biased by Brand Logos , 2012 .

[47]  Jerome R. Busemeyer,et al.  Using Cognitive Models to Map Relations Between Neuropsychological Disorders and Human Decision-Making Deficits , 2005, Psychological science.

[48]  Sarah E. MacPherson,et al.  Age, executive function, and social decision making: a dorsolateral prefrontal theory of cognitive aging. , 2002, Psychology and aging.

[49]  Eldad Yechiam,et al.  Decision making and personality in traffic offenders: a study of Israeli drivers. , 2008, Accident; analysis and prevention.

[50]  Keith E Stanovich,et al.  Decision-making and cognitive abilities: A review of associations between Iowa Gambling Task performance, executive functions, and intelligence. , 2010, Clinical psychology review.

[51]  J. Hsieh,et al.  Immediate gain is long-term loss: Are there foresighted decision makers in the Iowa Gambling Task? , 2008, Behavioral and Brain Functions.

[52]  Jerome R. Busemeyer,et al.  Evaluating generalizability and parameter consistency in learning models , 2008, Games Econ. Behav..

[53]  Simone G. Shamay-Tsoory,et al.  Adapted to explore: Reinforcement learning in Autistic Spectrum Conditions , 2010, Brain and Cognition.

[54]  E. Yechiam,et al.  Motivational processes and autonomic responsivity in Asperger's disorder: Evidence from the Iowa Gambling Task , 2006, Journal of the International Neuropsychological Society.

[55]  Timothy J. Pleskac,et al.  (www.interscience.wiley.com) DOI: 10.1002/bdm.641 Similar Processes Despite Divergent Behavior in Two Commonly Used Measures of Risky Decision Making , 2009 .

[56]  A. Damasio,et al.  Insensitivity to future consequences following damage to human prefrontal cortex , 1994, Cognition.

[57]  J. Busemeyer,et al.  Older adults as adaptive decision makers: evidence from the Iowa Gambling Task. , 2005, Psychology and aging.

[58]  Eldad Yechiam,et al.  Iowa Gambling Task in schizophrenia: A review and new data in patients with schizophrenia and co-occurring cannabis use disorders , 2007, Schizophrenia Research.

[59]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[60]  Scott D. Lane,et al.  Relationship between impulsivity and decision making in cocaine dependence , 2010, Psychiatry Research.

[61]  Eldad Yechiam,et al.  Application of a computational decision model to examine acute drug effects on human risk taking. , 2006, Experimental and clinical psychopharmacology.