Ensemble representation for multiple facial expressions: Evidence for a capacity limited perceptual process.

We tested the processing capacity of establishing ensemble representation for multiple facial expressions using the simultaneous-sequential paradigm. Each set consisted of 16 faces conveying a variable amount of happy and angry expressions. Participants judged on a continuous scale the perceived average emotion from each face set (Experiment 1). In the simultaneous condition, the 16 faces were presented concurrently; in the sequential condition, two sets, each containing eight faces, were presented successively. Results showed that judgments varied depending on the number of happy versus angry faces contained in the sets and were sensitive at the single trial level to the perceived mean emotion intensity (based on postexperiment ratings), providing evidence of a genuine mean representation rather than the mere use of a single face or enumeration. Experiments 2 and 3 replicated Experiment 1, but implemented a different response format (binary choices) and added masks following each display, respectively. Importantly, in all three experiments, performance was consistently better in the sequential than in the simultaneous condition, revealing a limited-capacity process. A set of control analyses ruled out the use of enumeration or mere subsampling by the participants to perform the task. Collectively, these results indicate that participants could readily extract mean emotion from multiple faces shown concurrently in a set, but this process is best conceived as being capacity limited.

[1]  L. Pessoa To what extent are emotional visual stimuli processed without attention and awareness? , 2005, Current Opinion in Neurobiology.

[2]  Jüri Allik,et al.  An almost general theory of mean size perception , 2013, Vision Research.

[3]  P. Ekman Facial expression and emotion. , 1993, The American psychologist.

[4]  Daniel J. Simons,et al.  Average size perception and the allure of a new mechanism , 2008 .

[5]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[6]  Xiaolan Fu,et al.  Processing of Individual Items during Ensemble Coding of Facial Expressions , 2016, Front. Psychol..

[7]  C. Chubb,et al.  A 'dipper' function for texture discrimination based on orientation variance. , 2008, Journal of vision.

[8]  C. Moore,et al.  Evidence of fixed capacity in visual object categorization , 2011, Psychonomic bulletin & review.

[9]  Steven L Franconeri,et al.  Selecting and tracking multiple objects. , 2015, Wiley interdisciplinary reviews. Cognitive science.

[10]  Jason M Haberman,et al.  From Textures to Crowds : Multiple Levels of Summary Statistical Perception , 2017 .

[11]  Mark H. Johnson Subcortical face processing , 2005, Nature Reviews Neuroscience.

[12]  Daniel J Simons,et al.  Ensemble representations: effects of set size and item heterogeneity on average size perception. , 2013, Acta psychologica.

[13]  Cathleen M Moore,et al.  Summary statistics of size: fixed processing capacity for multiple ensembles but unlimited processing capacity for single ensembles. , 2014, Journal of experimental psychology. Human perception and performance.

[14]  Jennifer E. Corbett,et al.  The whole is indeed more than the sum of its parts: perceptual averaging in the absence of individual item representation. , 2011, Acta psychologica.

[15]  G. Alvarez Representing multiple objects as an ensemble enhances visual cognition , 2011, Trends in Cognitive Sciences.

[16]  D. Natelson Correlated electron systems: Better than average. , 2009, Nature nanotechnology.

[17]  S. Dakin Information limit on the spatial integration of local orientation signals. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  Jason M Haberman,et al.  The visual system discounts emotional deviants when extracting average expression , 2010, Attention, perception & psychophysics.

[19]  Judith D. Singer,et al.  Using SAS PROC MIXED to Fit Multilevel Models, Hierarchical Models, and Individual Growth Models , 1998 .

[20]  A. Treisman,et al.  Representation of statistical properties , 2003, Vision Research.

[21]  Suk Won Han,et al.  Your own face is no more precious than others’: Evidence from the simultaneous–sequential paradigm , 2016, Psychonomic bulletin & review.

[22]  Cathleen M Moore,et al.  The perceptual processing capacity of summary statistics between and within feature dimensions. , 2015, Journal of vision.

[23]  Jason M Haberman,et al.  Efficient summary statistical representation when change localization fails , 2011, Psychonomic bulletin & review.

[24]  G. Rhodes Looking at Faces: First-Order and Second-Order Features as Determinants of Facial Appearance , 2013, Perception.

[25]  Joshua A Solomon,et al.  Efficiencies for the statistics of size discrimination. , 2011, Journal of vision.

[26]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[27]  N. Srinivasan,et al.  Processing statistics: An examination of focused and distributed attention using event related potentials , 2013, Vision Research.

[28]  A. Young,et al.  Understanding the recognition of facial identity and facial expression , 2005, Nature Reviews Neuroscience.

[29]  L. Pessoa,et al.  Emotion processing and the amygdala: from a 'low road' to 'many roads' of evaluating biological significance , 2010, Nature Reviews Neuroscience.

[30]  R. Shiffrin,et al.  Visual processing capacity and attentional control. , 1972, Journal of experimental psychology.

[31]  R. Dolan,et al.  A subcortical pathway to the right amygdala mediating "unseen" fear. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Bjorn Hubert-Wallander,et al.  Not all summary statistics are made equal: Evidence from extracting summaries across time. , 2015, Journal of vision.

[33]  John Palmer,et al.  Extending the simultaneous-sequential paradigm to measure perceptual capacity for features and words. , 2011, Journal of experimental psychology. Human perception and performance.

[34]  G. Pourtois,et al.  Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging , 2007, Neuropsychologia.

[35]  G. Pourtois,et al.  Mean emotion from multiple facial expressions can be extracted with limited attention: Evidence from visual ERPs , 2018, Neuropsychologia.

[36]  D. Whitney,et al.  Object-level visual information gets through the bottleneck of crowding. , 2011, Journal of neurophysiology.

[37]  David Whitney,et al.  Ensemble perception: Summarizing the scene and broadening the limits of visual processing. , 2012 .

[38]  P. Bex,et al.  Spatial frequency, phase, and the contrast of natural images. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[39]  C. Eriksen,et al.  Rate of information processing in visual perception: some results and methodological considerations. , 1969, Journal of experimental psychology.

[40]  D. Ariely Better than average? When can we say that subsampling of items is better than statistical summary representations? , 2008, Perception & psychophysics.

[41]  J. Tanaka,et al.  The NimStim set of facial expressions: Judgments from untrained research participants , 2009, Psychiatry Research.

[42]  D. Simons,et al.  Better than average: Alternatives to statistical summary representations for rapid judgments of average size , 2008, Perception & psychophysics.

[43]  Nicolas Robitaille,et al.  When more is less: extraction of summary statistics benefits from larger sets. , 2011, Journal of vision.

[44]  Steven C. Dakin,et al.  Local and global limitations on direction integration assessed using equivalent noise analysis , 2005, Vision Research.

[45]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[46]  Michael A. Cohen,et al.  What is the Bandwidth of Perceptual Experience? , 2016, Trends in Cognitive Sciences.

[47]  M. Tamietto,et al.  Neural bases of the non-conscious perception of emotional signals , 2010, Nature Reviews Neuroscience.

[48]  Daniel Smilek,et al.  Visual search for faces with emotional expressions. , 2008, Psychological bulletin.

[49]  Chris Oriet,et al.  Size and emotion averaging: costs of dividing attention after all. , 2012, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[50]  Murray B. Stein,et al.  Affective ambiguity for a group recruits ventromedial prefrontal cortex , 2006, NeuroImage.

[51]  Timothy F. Brady,et al.  Hierarchical Encoding in Visual Working Memory , 2010, Psychological science.

[52]  P. Vuilleumier,et al.  How brains beware: neural mechanisms of emotional attention , 2005, Trends in Cognitive Sciences.

[53]  Timothy F. Brady,et al.  Individual differences in ensemble perception reveal multiple, independent levels of ensemble representation. , 2015, Journal of experimental psychology. General.

[54]  Steven Dakin,et al.  Spatial limitations in averaging social cues , 2016, Scientific Reports.

[55]  A. Oliva,et al.  The Representation of Simple Ensemble Visual Features Outside the Focus of Attention , 2008, Psychological science.

[56]  I. Utochkin,et al.  Parallel averaging of size is possible but range-limited: a reply to Marchant, Simons, and De Fockert. , 2014, Acta psychologica.

[57]  D. Ariely Seeing Sets: Representation by Statistical Properties , 2001, Psychological science.

[58]  Aude Oliva,et al.  Spatial ensemble statistics are efficient codes that can be represented with reduced attention , 2009, Proceedings of the National Academy of Sciences.

[59]  L. Pessoa On the relationship between emotion and cognition , 2008, Nature Reviews Neuroscience.

[60]  Jason M Haberman,et al.  Seeing the mean: ensemble coding for sets of faces. , 2009, Journal of experimental psychology. Human perception and performance.

[61]  R. Marois,et al.  Capacity limits of information processing in the brain , 2005, Trends in Cognitive Sciences.

[62]  G. Kovács,et al.  Neural correlates of after-effects caused by adaptation to multiple face displays , 2012, Experimental Brain Research.

[63]  Ronald A. Rensink,et al.  Competition for consciousness among visual events: the psychophysics of reentrant visual processes. , 2000, Journal of experimental psychology. General.

[64]  S. Rauch,et al.  Masked Presentations of Emotional Facial Expressions Modulate Amygdala Activity without Explicit Knowledge , 1998, The Journal of Neuroscience.

[65]  Sung Jun Joo,et al.  Statistical processing: Not so implausible after all , 2008, Perception & Psychophysics.

[66]  J. Lund,et al.  Compulsory averaging of crowded orientation signals in human vision , 2001, Nature Neuroscience.

[67]  Jin Fan,et al.  Searching for the Majority: Algorithms of Voluntary Control , 2008, PloS one.

[68]  Jason M Haberman,et al.  Correspondences Rapid extraction of mean emotion and gender from sets of faces , 2007 .

[69]  S. C. Chong,et al.  Accurate but Pathological: Social Anxiety and Ensemble Coding of Emotion , 2013, Cognitive Therapy and Research.

[70]  H. Akaike A new look at the statistical model identification , 1974 .

[71]  C. Moore,et al.  The capacity limitations of orientation summary statistics , 2015, Attention, perception & psychophysics.

[72]  Yuhong V. Jiang,et al.  Redundancy Effects in the Processing of Emotional Faces , 2011 .

[73]  Z. Pylyshyn,et al.  Why are small and large numbers enumerated differently? A limited-capacity preattentive stage in vision. , 1994, Psychological review.