The problem state bottleneck: modeling the behavoral and neural signatures of a cognitive bottleneck in human multitasking

Deze samenvatting, en ook grote delen van dit proefschrift, zijn geschreven met behulp van het programma ‘Concen- trate’1. Concentrate is geen normaal programma, het kan niet gebruikt worden om mee te chatten, te schrijven, of te internetten. Nee, Concentrate zorgt er juist voor dat er niet teveel dingen tegelijkertijd gedaan kunnen worden, het dwingt monotasking af. Want terwijl in onze samenleving multitasken langzamerhand tot de standaard wordt verheven – hoe vaak zie je wel geen bellende fietsers? – hebben wetenschappers juist laten zien dat mensen vaak minder goed presteren als ze meerdere dingen tegelijkertijd doen. Dit proefschrift gaat over een van de oorzaken van multitaskingproblemen: een beperking in het verwerken van tussenresultaten (bijvoorbeeld ‘3x = 12’ wanneer ‘3x – 7 = 5’ opgelost moet worden) . We laten zien dat als mensen een tussenresultaat voor meerdere taken tegelijkertijd moeten onthouden, hun prestatie op deze taken sterk vermindert. Volgens onze theorie worden de tussenresultaten namelijk opgeslagen in de zogeheten werkgeheugen-module in ons brein. Deze module kan echter maar een tussenresultaat tegelijkertijd opslaan, en fungeert dus als een beperkende factor in het multitasken wanneer mensen meerder tussenresultaten nodig hebben. Als dat het geval is moeten de tussenresultaten namelijk constant uitgewisseld worden tussen de werkgeheugen-module en het gewone geheugen. Deze uitwisseling kost tijd en kan soms misgaan, en leidt daarom tot een verminderde multitaskingprestatie. In deze samenvatting zal ik proberen duidelijk te maken hoe we deze beperking in het verwerken van tussenresultaten (door ons de problem state bottleneck genoemd) onderzocht hebben, en wat de specifieke resultaten waren. Ik zal eerst de onderzoeksmethoden toelichten die we gebruikt hebben, gevolgd door de resultaten en onze uiteindelijke theorie.

[1]  Karl J. Friston,et al.  Statistical parametric mapping , 2013 .

[2]  Nelson Cowan,et al.  When visual and verbal memories compete: Evidence of cross-domain limits in working memory , 2004, Psychonomic bulletin & review.

[3]  Myeong-Ho Sohn,et al.  An information-processing model of three cortical regions: evidence in episodic memory retrieval , 2005, NeuroImage.

[4]  Michael K. Martin,et al.  Instance-Based Decision Making Model of Repeated Binary Choice , 2007 .

[5]  Peter A. Bandettini,et al.  From neuron to BOLD: new connections , 2001, Nature Neuroscience.

[6]  E. Hess,et al.  Pupil Size in Relation to Mental Activity during Simple Problem-Solving , 1964, Science.

[7]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[8]  Sander Martens,et al.  A quick visual mind can be a slow auditory mind. Individual differences in attentional selection across modalities. , 2009, Experimental psychology.

[9]  K A Brookhuis,et al.  The effects of mobile telephoning on driving performance. , 1991, Accident; analysis and prevention.

[10]  J. C. Johnston,et al.  Attentional limitations in dual-task performance. , 1998 .

[11]  E E Smith,et al.  Components of verbal working memory: evidence from neuroimaging. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[12]  I. J. Myung,et al.  When a good fit can be bad , 2002, Trends in Cognitive Sciences.

[13]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[14]  John R. Anderson,et al.  Serial modules in parallel: the psychological refractory period and perfect time-sharing. , 2001, Psychological review.

[15]  Wayne D. Gray,et al.  The soft constraints hypothesis: a rational analysis approach to resource allocation for interactive behavior. , 2006, Psychological review.

[16]  F. Zijlstra,et al.  Temporal factors in mental work: Effects of interrupted activities , 1999 .

[17]  Hugh Garavan,et al.  Serial attention within working memory , 1998, Memory & cognition.

[18]  Dario D. Salvucci,et al.  Multitasking and monotasking: the effects of mental workload on deferred task interruptions , 2010, CHI.

[19]  Wayne D. Gray,et al.  An integrated model of cognitive control in task switching. , 2008, Psychological review.

[20]  Wayne D. Gray Cognitive Architectures: Choreographing the Dance of Mental Operations With the Task Environment , 2008, Hum. Factors.

[21]  I. J. Myung,et al.  The Importance of Complexity in Model Selection. , 2000, Journal of mathematical psychology.

[22]  A. Welford THE ‘PSYCHOLOGICAL REFRACTORY PERIOD’ AND THE TIMING OF HIGH‐SPEED PERFORMANCE—A REVIEW AND A THEORY , 1952 .

[23]  Jennifer Spenader,et al.  Modeling the selective effects of slowed-down speech in pronoun comprehension. , 2009 .

[24]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[25]  N. Logothetis What we can do and what we cannot do with fMRI , 2008, Nature.

[26]  D. Schacter,et al.  Prefrontal Contributions to Executive Control: fMRI Evidence for Functional Distinctions within Lateral Prefrontal Cortex , 2001, NeuroImage.

[27]  M. Coltheart What has Functional Neuroimaging told us about the Mind (so far)? (Position Paper Presented to the European Cognitive Neuropsychology Workshop, Bressanone, 2005) , 2006, Cortex.

[28]  Eliot Hazeltine,et al.  Target selection during bimanual reaching to direct cues is unaffected by the perceptual similarity of the targets. , 2007, Journal of experimental psychology. Human perception and performance.

[29]  John R. Anderson How Can the Human Mind Occur in the Physical Universe , 2007 .

[30]  John R. Anderson,et al.  Learning to achieve perfect timesharing: architectural implications of Hazeltine, Teague, and Ivry (2002). , 2005, Journal of experimental psychology. Human perception and performance.

[31]  G. Woodman,et al.  Visual Search Remains Efficient when Visual Working Memory is Full , 2001, Psychological science.

[32]  Mitsuo Kawato,et al.  Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning , 2006, Neural Networks.

[33]  David E. Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 2. Accounts of psychological refractory-period phenomena. , 1997 .

[34]  John R. Anderson,et al.  Constraints in Cognitive Architectures , 2008 .

[35]  E. M. Altmann,et al.  Timecourse of recovery from task interruption: Data and a model , 2006, Psychonomic bulletin & review.

[37]  David E. Kieras,et al.  An Overview of the EPIC Architecture for Cognition and Performance With Application to Human-Computer Interaction , 1997, Hum. Comput. Interact..

[38]  John R. Anderson,et al.  Human Symbol Manipulation Within an Integrated Cognitive Architecture , 2005, Cogn. Sci..

[39]  Kristen L. Macuga,et al.  Predicting the effects of cellular-phone dialing on driver performance , 2002, Cognitive Systems Research.

[40]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[41]  John R. Anderson,et al.  Tracing Problem Solving in Real Time: fMRI Analysis of the Subject-paced Tower of Hanoi , 2005, Journal of Cognitive Neuroscience.

[42]  N. Taatgen,et al.  The problem state: a cognitive bottleneck in multitasking. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[43]  G. Glover Deconvolution of Impulse Response in Event-Related BOLD fMRI1 , 1999, NeuroImage.

[44]  Richard L. Lewis,et al.  Computational principles of working memory in sentence comprehension , 2006, Trends in Cognitive Sciences.

[45]  Richard L. Lewis,et al.  Rational adaptation under task and processing constraints: implications for testing theories of cognition and action. , 2009, Psychological review.

[46]  Eliot Hazeltine,et al.  Simultaneous dual-task performance reveals parallel response selection after practice. , 2002, Journal of experimental psychology. Human perception and performance.

[47]  Richard P. Cooper,et al.  The Role of Falsification in the Development of Cognitive Architectures: Insights from a Lakatosian Analysis , 2007, Cogn. Sci..

[48]  N. Daw,et al.  Reinforcement learning and higher level cognition: Introduction to special issue , 2009, Cognition.

[49]  Klaus Oberauer,et al.  Design for a working memory. , 2009 .

[50]  Marsha C. Lovett,et al.  A source activation theory of working memory: cross-task prediction of performance in ACT-R , 2000, Cognitive Systems Research.

[51]  M. Hogge,et al.  Exploration of the neural substrates of executive functioning by functional neuroimaging , 2006, Neuroscience.

[52]  J R Anderson,et al.  Task preparation and task repetition: two-component model of task switching. , 2001, Journal of experimental psychology. General.

[53]  Christopher L. Asplund,et al.  Isolation of a Central Bottleneck of Information Processing with Time-Resolved fMRI , 2006, Neuron.

[54]  Andrea Stocco,et al.  The Neural Correlates of Problem States: Testing fMRI Predictions of a Computational Model of Multitasking , 2010, PloS one.

[55]  K. Oberauer Access to information in working memory: exploring the focus of attention. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[56]  Mark D'Esposito,et al.  Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses , 2004, NeuroImage.

[57]  J. O'Doherty,et al.  Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain , 2006, PLoS biology.

[58]  Dario D. Salvucci A Multitasking General Executive for Compound Continuous Tasks , 2005, Cogn. Sci..

[59]  Petra Hendriks,et al.  Learning to reason about speakers' alternatives in sentence comprehension: A computational account , 2007 .

[60]  Steven W. Keele,et al.  Attention and human performance , 1973 .

[61]  A. Dale,et al.  Selective averaging of rapidly presented individual trials using fMRI , 1997, Human brain mapping.

[62]  G. Logan,et al.  On the Use of a Concurrent Memory Load to Measure Attention and Automaticity , 1979 .

[63]  Cameron S. Carter,et al.  The Relationship of Three Cortical Regions to an Information-Processing Model , 2004, Journal of Cognitive Neuroscience.

[64]  Yili Liu,et al.  Queueing Network-Model Human Processor (QN-MHP): A computational architecture for multitask performance in human-machine systems , 2006, TCHI.

[65]  Florin Dolcos,et al.  Similarities and Differences in the Neural Correlates of Episodic Memory Retrieval and Working Memory , 2002, NeuroImage.

[66]  Richard Henson,et al.  What has (Neuro)Psychology told us About the Mind (so Far)? a Reply to Coltheart (2006) , 2006, Cortex.

[67]  Jelmer P. Borst,et al.  The Costs of Multitasking in Threaded Cognition , 2007 .

[68]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[69]  D. Noll,et al.  Nonlinear Aspects of the BOLD Response in Functional MRI , 1998, NeuroImage.

[70]  John T Cacioppo,et al.  Great expectations: what can fMRI research tell us about psychological phenomena? , 2009, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[71]  Klaus Oberauer,et al.  Accessing information in working memory: can the focus of attention grasp two elements at the same time? , 2009, Journal of experimental psychology. General.

[72]  Karl J. Friston Modalities, Modes, and Models in Functional Neuroimaging , 2009, Science.

[73]  J. Gregory Trafton,et al.  Memory for goals: an activation-based model , 2002, Cogn. Sci..

[74]  Niels Taatgen,et al.  Modeling Triple-Tasking without Customized Cognitive Control , 2009 .

[75]  John R. Anderson,et al.  Endogenous Control and Task Representation: An fMRI Study in Algebraic Problem-solving , 2008, Journal of Cognitive Neuroscience.

[76]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[77]  B. Ross The Psychology of Learning and Motivation , 2012 .

[78]  H Pashler,et al.  Processing stages in overlapping tasks: evidence for a central bottleneck. , 1984, Journal of experimental psychology. Human perception and performance.

[79]  M. Just,et al.  A decrease in brain activation associated with driving when listening to someone speak , 2008, Brain Research.

[80]  D. Kahneman,et al.  Attention and Effort , 1973 .

[81]  Max Coltheart,et al.  Brain Imaging, Connectionism, and Cognitive Neuropsychology , 2004, Cognitive neuropsychology.

[82]  N. Cowan The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[83]  S. Steinhauer,et al.  The Pupillary Response in Cognitive Psychophysiology and Schizophrenia a , 1992, Annals of the New York Academy of Sciences.

[84]  R. Cabeza,et al.  Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies , 2000, Journal of Cognitive Neuroscience.

[85]  A. Newell You can't play 20 questions with nature and win : projective comments on the papers of this symposium , 1973 .

[86]  John R. Anderson,et al.  Using Brain Imaging to Extract the Structure of Complex Events at the Rational Time Band , 2008, Journal of Cognitive Neuroscience.

[87]  Xenophon Papademetris,et al.  More accurate Talairach coordinates for neuroimaging using non-linear registration , 2008, NeuroImage.

[88]  Larry D. Rosen,et al.  Multitasking across generations: Multitasking choices and difficulty ratings in three generations of Americans , 2009, Comput. Hum. Behav..

[89]  Richard Reviewer-Granger Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[90]  P. Hagoort,et al.  Should Psychology Ignore the Language of the Brain ? , 2008 .

[91]  Petra Hendriks,et al.  Cognitive architectures and language acquisition: a case study in pronoun comprehension. , 2010, Journal of child language.

[92]  Allen Newell,et al.  Physical Symbol Systems , 1980, Cogn. Sci..

[93]  Jean-Luc Anton,et al.  Region of interest analysis using an SPM toolbox , 2010 .

[94]  A. Baddeley The episodic buffer: a new component of working memory? , 2000, Trends in Cognitive Sciences.

[95]  T. Harley Does Cognitive Neuropsychology have a future? , 2004, Cognitive neuropsychology.

[96]  Niels Taatgen,et al.  Traces of times past: Representations of temporal intervals in memory , 2011, Memory & cognition.

[97]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[98]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982, Psychological bulletin.

[99]  R. Henson What can Functional Neuroimaging Tell the Experimental Psychologist? , 2005, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[100]  Niels Taatgen,et al.  Using a symbolic process model as input for model-based fMRI analysis: Locating the neural correlates of problem state replacements , 2011, NeuroImage.

[101]  Dario D. Salvucci An integrated model of eye movements and visual encoding , 2001, Cognitive Systems Research.

[102]  Roberto Dell'Acqua,et al.  Attentional and structural constraints on visual encoding , 1999 .

[103]  Marsha C. Lovett,et al.  Modeling individual differences in working memory performance: a source activation account , 2001, Cogn. Sci..

[104]  Richard L. Lewis,et al.  An Activation-Based Model of Sentence Processing as Skilled Memory Retrieval , 2005, Cogn. Sci..

[105]  N. Taatgen,et al.  Timing of multiple overlapping intervals: how many clocks do we have? , 2008, Acta psychologica.

[106]  Daniel Gopher,et al.  On the Economy of the Human Processing System: A Model of Multiple Capacity. , 1977 .

[107]  Karl J. Friston,et al.  Nonlinear Regression in Parametric Activation Studies , 1996, NeuroImage.

[108]  Richard L. Lewis,et al.  The mind and brain of short-term memory. , 2008, Annual review of psychology.

[109]  Christopher D Wickens,et al.  Processing Resources in Attention, Dual Task Performance, and Workload Assessment. , 1981 .

[110]  Niels Taatgen,et al.  Modeling Parallelization and Flexibility Improvements in Skill Acquisition: From Dual Tasks to Complex Dynamic Skills , 2005, Cogn. Sci..

[111]  N. Taatgen,et al.  Too much control can hurt: A threaded cognition model of the attentional blink , 2009, Cognitive Psychology.

[112]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[113]  Dylan M. Jones,et al.  Interruption of the Tower of London task: support for a goal-activation approach. , 2006, Journal of experimental psychology. General.

[114]  John R. Anderson,et al.  A central circuit of the mind , 2008, Trends in Cognitive Sciences.

[115]  D. Broadbent Perception and communication , 1958 .

[116]  James D. Hollan,et al.  Distributed cognition: toward a new foundation for human-computer interaction research , 2000, TCHI.

[117]  Dirk van Rijn,et al.  Testing fMRI Predictions of a Dual-Task Interference Model , 2009 .

[118]  Niels Taatgen,et al.  Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption , 2009, CHI.

[119]  Trudy Buwalda,et al.  Evading a Multitasking Bottleneck: Presenting Intermediate Representations in the Environment , 2011, CogSci.

[120]  J. Trafton,et al.  The effect of interruption duration and demand on resuming suspended goals. , 2008, Journal of experimental psychology. Applied.

[121]  Niels Taatgen,et al.  The Multitasking Mind , 2010, Oxford series on cognitive models and architectures.

[122]  C. Wickens Engineering psychology and human performance, 2nd ed. , 1992 .

[123]  Jelmer P. Borst,et al.  Memory-decay problems in a level-repetition switch task model , 2006 .

[124]  Dario D. Salvucci,et al.  Threaded cognition: an integrated theory of concurrent multitasking. , 2008, Psychological review.

[125]  S. Lewandowsky,et al.  Forgetting in immediate serial recall: decay, temporal distinctiveness, or interference? , 2008, Psychological review.

[126]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[127]  John R. Anderson,et al.  Reflections of the Environment in Memory Form of the Memory Functions , 2022 .

[128]  H Pashler,et al.  How persuasive is a good fit? A comment on theory testing. , 2000, Psychological review.

[129]  H. Pashler Dual-task interference in simple tasks: data and theory. , 1994, Psychological bulletin.

[130]  Klaus Wunderlich,et al.  Neural computations underlying action-based decision making in the human brain , 2009, Proceedings of the National Academy of Sciences.

[131]  Jelmer P. Borst,et al.  Stroop and picture—word interference are two sides of the same coin , 2009, Psychonomic bulletin & review.

[132]  N. Cowan Attention and Memory: An Integrated Framework , 1995 .

[133]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[134]  Kara A. Latorella,et al.  The Scope and Importance of Human Interruption in Human-Computer Interaction Design , 2002, Hum. Comput. Interact..

[135]  Dario D. Salvucci Modeling Driver Behavior in a Cognitive Architecture , 2006, Hum. Factors.

[136]  B McElree,et al.  Working memory and focal attention. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[137]  Edward E. Smith,et al.  Temporal dynamics of brain activation during a working memory task , 1997, Nature.

[138]  Christian Lebiere,et al.  The dynamics of cognition: An ACT-R model of cognitive arithmetic , 1999, Kognitionswissenschaft.

[139]  T. Shallice,et al.  Task Switching : A PDP Model , 2001 .

[140]  N. Kanwisher,et al.  Neuroimaging of cognitive functions in human parietal cortex , 2001, Current Opinion in Neurobiology.

[141]  Christopher A. Monk,et al.  A Process-Model Account of Task Interruption and Resumption: When Does Encoding of the Problem State Occur? , 2009 .

[142]  F. Collette,et al.  Brain imaging of the central executive component of working memory , 2002, Neuroscience & Biobehavioral Reviews.

[143]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience , 2009 .

[144]  K. R. Ridderinkhof,et al.  The Role of the Medial Frontal Cortex in Cognitive Control , 2004, Science.

[145]  J. O'Doherty,et al.  Model‐Based fMRI and Its Application to Reward Learning and Decision Making , 2007, Annals of the New York Academy of Sciences.

[146]  D. Norman,et al.  Attention to Action: Willed and Automatic Control of Behavior Technical Report No. 8006. , 1980 .

[147]  D. Kieras,et al.  Modern computational perspectives on executive mental processes and cognitive control: Where to from here? , 2000 .

[148]  S. Dehaene,et al.  THREE PARIETAL CIRCUITS FOR NUMBER PROCESSING , 2003, Cognitive neuropsychology.

[149]  Brian P. Bailey,et al.  Towards an index of opportunity: understanding changes in mental workload during task execution , 2004, CHI.

[150]  M. Page,et al.  What Can't Functional Neuroimaging Tell the Cognitive Psychologist? , 2006, Cortex.

[151]  Mark S. Cohen,et al.  Parametric Analysis of fMRI Data Using Linear Systems Methods , 1997, NeuroImage.

[152]  Víctor M. González,et al.  "Constant, constant, multi-tasking craziness": managing multiple working spheres , 2004, CHI.

[153]  Jennifer M. Glass,et al.  Virtually Perfect Time Sharing in Dual-Task Performance: Uncorking the Central Cognitive Bottleneck , 2001, Psychological science.

[154]  W. Kirchner Age differences in short-term retention of rapidly changing information. , 1958, Journal of experimental psychology.

[155]  John P O'Doherty,et al.  Model-based approaches to neuroimaging: combining reinforcement learning theory with fMRI data. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[156]  S. Petersen,et al.  Frontal cortex contributes to human memory formation , 1999, Nature Neuroscience.

[157]  A. Baddeley Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[158]  J. Gregory Trafton,et al.  Preparing to resume an interrupted task: effects of prospective goal encoding and retrospective rehearsal , 2003, Int. J. Hum. Comput. Stud..

[159]  T. Shallice,et al.  CONTENTION SCHEDULING AND THE CONTROL OF ROUTINE ACTIVITIES , 2000, Cognitive neuropsychology.

[160]  Wai-Tat Fu,et al.  Soft constraints in interactive behavior: the case of ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head , 2004, Cogn. Sci..

[161]  D. Gitelman,et al.  Neuroanatomic Overlap of Working Memory and Spatial Attention Networks: A Functional MRI Comparison within Subjects , 1999, NeuroImage.

[162]  An ACT-R / PM Model of the Articulatory Loop , 2002 .

[163]  Niels Taatgen,et al.  Using a Model-Based fMRI Analysis Method to Locate the Neural Correlates of a Multitasking Bottleneck , 2011, CogSci.

[164]  Niels Taatgen,et al.  Locating the neural correlates of the problem state resource : Analyzing fMRI data on the basis of a computational model , 2010 .

[165]  R. Henson,et al.  Frontal lobes and human memory: insights from functional neuroimaging. , 2001, Brain : a journal of neurology.

[166]  John R. Anderson Spanning seven orders of magnitude: a challenge for cognitive modeling , 2002 .

[167]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[168]  G. D. Logan Task Switching , 2022 .