Building Blocks of Visual Working Memory: Objects or Boolean Maps?

The nature of the building blocks of information in visual working memory (VWM) is a fundamental issue that has not been well resolved. Most researchers take objects as the building blocks, although this perspective has received criticism. The objects could be physically separated ones (strict object hypothesis) or hierarchical objects created from separated individuals (broad object hypothesis). Meanwhile, a newly proposed Boolean map theory for visual attention suggests that Boolean maps may be the building blocks of VWM (Boolean map hypothesis); this perspective could explain many critical findings of VWM. However, no previous study has examined these hypotheses. We explored this issue by focusing on a critical point on which they make distinct predictions. We asked participants to remember two distinct objects (2-object), three distinct objects (3-object), or three objects with repeated information (mixed-3-object, e.g., one red bar and two green bars, green bars could be represented as one hierarchical object) and adopted contralateral delay activity (CDA) to tap into the maintenance phase of VWM. The mixed-3-object condition could generate two Boolean maps, three objects, or three objects most of the time (hierarchical objects are created in certain trials, retaining two objects). Simple orientations (Experiment 1) and colors (Experiments 2 and 3) were used as stimuli. Although the CDA of the mixed-3-object condition was slightly lower than that of the 3-object condition, no significant difference was revealed between them. Both conditions displayed significantly higher CDAs than the 2-object condition. These findings support the broad object hypothesis. We further suggest that Boolean maps might be the unit for retrieval/comparison in VWM.

[1]  Paul M Bays,et al.  Dynamic Shifts of Limited Working Memory Resources in Human Vision , 2008, Science.

[2]  G. Woodman,et al.  The comparison of visual working memory representations with perceptual inputs. , 2009, Journal of experimental psychology. Human perception and performance.

[3]  M. Chun,et al.  Dissociable neural mechanisms supporting visual short-term memory for objects , 2006, Nature.

[4]  E. Vogel,et al.  Contralateral delay activity provides a neural measure of the number of representations in visual working memory. , 2010, Journal of neurophysiology.

[5]  Edward Awh,et al.  Selection and storage of perceptual groups is constrained by a discrete resource in working memory. , 2013, Journal of experimental psychology. Human perception and performance.

[6]  Steven J. Luck,et al.  The Number and Quality of Representations in Working Memory , 2011, Psychological science.

[7]  Pieter R Roelfsema,et al.  Matching of visual input to only one item at any one time , 2009, Psychological research.

[8]  E. Vogel,et al.  Capacity limit of visual short-term memory in human posterior parietal cortex , 2004 .

[9]  M. Shen,et al.  Contralateral delay activity tracks object identity information in visual short term memory , 2011, Brain Research.

[10]  M. Shen,et al.  Tracking object number or information load in visual working memory: Revisiting the cognitive implication of contralateral delay activity , 2011, Biological Psychology.

[11]  Amanda L. Gilchrist,et al.  Seven-year-olds allocate attention like adults unless working memory is overloaded. , 2010, Developmental science.

[12]  Maro G. Machizawa,et al.  Neural activity predicts individual differences in visual working memory capacity , 2004, Nature.

[13]  G. Alvarez,et al.  Object features fail independently in visual working memory: evidence for a probabilistic feature-store model. , 2011, Journal of vision.

[14]  Timothy F. Brady,et al.  A review of visual memory capacity: Beyond individual items and toward structured representations. , 2011, Journal of vision.

[15]  Edward K. Vogel,et al.  Shape and color conjunction stimuli are represented as bound objects in visual working memory , 2011 .

[16]  E. Vogel,et al.  Interactions between attention and working memory , 2006, Neuroscience.

[17]  Roberto Cubelli,et al.  Lack of color integration in visual short-term memory binding , 2011, Memory & cognition.

[18]  R. Logie,et al.  Specific deficit of colour–colour short-term memory binding in sporadic and familial Alzheimer's disease , 2011, Neuropsychologia.

[19]  Wei Ji Ma,et al.  Variability in encoding precision accounts for visual short-term memory limitations , 2012, Proceedings of the National Academy of Sciences.

[20]  Lester C. Loschky,et al.  Do object refixations during scene viewing indicate rehearsal in visual working memory? , 2011, Memory & cognition.

[21]  N. Cowan The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[22]  Liqiang Huang,et al.  Characterizing the Nature of Visual Conscious Access: The Distinction between Features and Locations , 2010, Journal of vision.

[23]  Yaoda Xu Understanding the object benefit in visual short-term memory: The roles of feature proximity and connectedness , 2006, Perception & Psychophysics.

[24]  Roy Luria,et al.  Visual Short-term Memory Capacity for Simple and Complex Objects , 2010, Journal of Cognitive Neuroscience.

[25]  Maro G. Machizawa,et al.  Neural measures reveal individual differences in controlling access to working memory , 2005, Nature.

[26]  Yaoda Xu,et al.  Visual grouping in human parietal cortex , 2007, Proceedings of the National Academy of Sciences.

[27]  Geoffrey F Woodman,et al.  Perceptual organization influences visual working memory , 2003, Psychonomic bulletin & review.

[28]  E. Vogel,et al.  Shape and color conjunction stimuli are represented as bound objects in visual working memory , 2011, Neuropsychologia.

[29]  G. Alvarez Representing multiple objects as an ensemble enhances visual cognition , 2011, Trends in Cognitive Sciences.

[30]  A. Baddeley Working memory: theories, models, and controversies. , 2012, Annual review of psychology.

[31]  Yuhong V. Jiang,et al.  Is visual short-term memory object based? Rejection of the “strong-object” hypothesis , 2002, Perception & psychophysics.

[32]  A. Treisman,et al.  Binding in short-term visual memory. , 2002, Journal of experimental psychology. General.

[33]  Daeyeol Lee,et al.  What are the units of visual short-term memory, objects or spatial locations? , 2001, Perception & psychophysics.

[34]  Rainer Goebel,et al.  Common neural substrates for visual working memory and attention , 2007, NeuroImage.

[35]  Edward K Vogel,et al.  Flexibility in visual working memory: Accurate change detection in the face of irrelevant variations in position , 2012, Visual cognition.

[36]  Mowei Shen,et al.  The neural mechanisms of percept–memory comparison in visual working memory , 2012, Biological Psychology.

[37]  Benoit Brisson,et al.  Dissociation of the N2pc and sustained posterior contralateral negativity in a choice response task , 2008, Brain Research.

[38]  Nelson Cowan,et al.  Visual working memory deficits in patients with Parkinson's disease are due to both reduced storage capacity and impaired ability to filter out irrelevant information. , 2010, Brain : a journal of neurology.

[39]  G. Woodman,et al.  The time course of consolidation in visual working memory. , 2006, Journal of experimental psychology. Human perception and performance.

[40]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[41]  Emma Y. Wu,et al.  Storage and binding of object features in visual working memory , 2011, Neuropsychologia.

[42]  M. Chun,et al.  Selecting and perceiving multiple visual objects , 2009, Trends in Cognitive Sciences.

[43]  Maro G. Machizawa,et al.  Electrophysiological Measures of Maintaining Representations in Visual Working Memory , 2007, Cortex.

[44]  Mowei Shen,et al.  The perceptual root of object-based storage: an interactive model of perception and visual working memory. , 2011, Journal of experimental psychology. Human perception and performance.

[45]  G. Woodman,et al.  Storage of features, conjunctions and objects in visual working memory. , 2001, Journal of experimental psychology. Human perception and performance.

[46]  H. Pashler,et al.  Distinguishing different strategies of across-dimension attentional selection. , 2012, Journal of experimental psychology. Human perception and performance.

[47]  G. Woodman,et al.  Selective storage and maintenance of an object’s features in visual working memory , 2008, Psychonomic bulletin & review.

[48]  Yaoda Xu,et al.  Distinctive Neural Mechanisms Supporting Visual Object Individuation and Identification , 2009, Journal of Cognitive Neuroscience.

[49]  Leo Poom,et al.  Visual working memory capacity and stimulus categories: a behavioral and electrophysiological investigation , 2011, Experimental Brain Research.

[50]  E. Vogel,et al.  Discrete capacity limits in visual working memory , 2010, Current Opinion in Neurobiology.

[51]  C. Caltagirone,et al.  Remembering What But Not Where: Independence of Spatial and Visual Working Memory in the Human Brain , 2001, Cortex.

[52]  Harold Pashler,et al.  A Boolean map theory of visual attention. , 2007, Psychological review.

[53]  Yaoda Xu,et al.  Limitations of object-based feature encoding in visual short-term memory. , 2002, Journal of experimental psychology. Human perception and performance.

[54]  Mowei Shen,et al.  Storing fine detailed information in visual working memory--evidence from event-related potentials. , 2009, Journal of vision.

[55]  Mowei Shen,et al.  Dissociated Mechanisms of Extracting Perceptual Information into Visual Working Memory , 2010, PloS one.

[56]  R. Kesner,et al.  The Effects of Prelimbic and Infralimbic Lesions on Working Memory for Visual Objects in Rats , 2002, Neurobiology of Learning and Memory.

[57]  Harold Pashler,et al.  Characterizing the Limits of Human Visual Awareness , 2007, Science.

[58]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[59]  Liqiang Huang The speed of feature-based attention: attentional advantage is slow, but selection is fast. , 2010, Journal of experimental psychology. Human perception and performance.

[60]  E. Vogel,et al.  Are old adults just like low working memory young adults? Filtering efficiency and age differences in visual working memory. , 2011, Cerebral cortex.

[61]  S. Luck,et al.  Discrete fixed-resolution representations in visual working memory , 2008, Nature.

[62]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[63]  M. Chun,et al.  Organization of visual short-term memory. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[64]  E. Vogel,et al.  PSYCHOLOGICAL SCIENCE Research Article Visual Working Memory Represents a Fixed Number of Items Regardless of Complexity , 2022 .

[65]  Liqiang Huang,et al.  What is the unit of visual attention? Object for selection, but Boolean map for access. , 2010, Journal of experimental psychology. General.