Integrative processing in artificial and biological vision predicts the perceived beauty of natural images

Previous research indicates that the beauty of natural images is already determined during perceptual analysis. However, it is still largely unclear which perceptual computations give rise to the perception of beauty. Theories of processing fluency suggest that the ease of processing for an image determines its perceived beauty. Here, we tested whether perceived beauty is related to the amount of spatial integration across an image, a perceptual computation that reduces processing demands by aggregating image elements into more efficient representations of the whole. We hypothesized that higher degrees of integration reduce processing demands in the visual system and thereby predispose the perception of beauty. We quantified integrative processing in an artificial deep neural network model of vision: We compared activations between parts of the image and the whole image, where the degree of integration was determined by the amount of deviation between activations for the whole image and its constituent parts. This quantification of integration predicted the beauty ratings for natural images across four studies, which featured different stimuli and task demands. In a complementary fMRI study, we show that integrative processing in human visual cortex predicts perceived beauty in a similar way as in artificial neural networks. Together, our results establish integration as a computational principle that facilitates perceptual analysis and thereby mediates the perception of beauty.

[1]  Felix Wichmann,et al.  Are Deep Neural Networks Adequate Behavioural Models of Human Visual Perception? , 2023, Annual review of vision science.

[2]  Radoslaw Martin Cichy,et al.  Alpha-frequency feedback to early visual cortex orchestrates coherent natural vision , 2023 .

[3]  Iman A. Wahle,et al.  Neural mechanisms underlying the hierarchical construction of perceived aesthetic value , 2023, Nature Communications.

[4]  M. Chun,et al.  Representing Multiple Visual Objects in the Human Brain and Convolutional Neural Networks , 2022, bioRxiv.

[5]  Chris I Tsvetkov,et al.  Deep problems with neural network models of human vision , 2022, Behavioral and Brain Sciences.

[6]  Daniel Kaiser.  Spectral brain signatures of aesthetic natural perception in the alpha and beta frequency bands , 2022, bioRxiv.

[7]  Daniel Kaiser.  Characterizing dynamic neural representations of scene attractiveness , 2022, bioRxiv.

[8]  Radoslaw Martin Cichy,et al.  Parts and Wholes in Scene Processing , 2021, Journal of Cognitive Neuroscience.

[9]  Edward A. Vessel,et al.  From Visual Perception to Aesthetic Appeal: Brain Responses to Aesthetically Appealing Natural Landscape Movies , 2021, Frontiers in Human Neuroscience.

[10]  Taylor R. Hayes,et al.  Looking for Semantic Similarity: What a Vector-Space Model of Semantics Can Tell Us About Attention in Real-World Scenes , 2021, Psychological science.

[11]  Yaoda Xu,et al.  Limits to visual representational correspondence between convolutional neural networks and the human brain , 2021, Nature Communications.

[12]  Harish Katti,et al.  Qualitative similarities and differences in visual object representations between brains and deep networks , 2021, Nature Communications.

[13]  Eghbal A. Hosseini,et al.  The neural architecture of language: Integrative modeling converges on predictive processing , 2020, Proceedings of the National Academy of Sciences.

[14]  Russell A. Epstein,et al.  Object representations in the human brain reflect the co-occurrence statistics of vision and language , 2020, Nature Communications.

[15]  Grace W. Lindsay.  Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future , 2020, Journal of Cognitive Neuroscience.

[16]  A. Chatterjee,et al.  The neural mechanism of aesthetic judgments of dynamic landscapes: an fMRI study , 2020, Scientific Reports.

[17]  Johan Wagemans,et al.  Order, complexity, and aesthetic appreciation. , 2020 .

[18]  Radoslaw Martin Cichy,et al.  Rapid contextualization of fragmented scene information in the human visual system , 2020, NeuroImage.

[19]  Galit Yovel,et al.  The Functional Organization of High-Level Visual Cortex Determines the Representation of Complex Visual Stimuli , 2019, The Journal of Neuroscience.

[20]  D. Pelli,et al.  Intense Beauty Requires Intense Pleasure , 2019, Front. Psychol..

[21]  Nikolaus Kriegeskorte,et al.  Recurrence is required to capture the representational dynamics of the human visual system , 2019, Proceedings of the National Academy of Sciences.

[22]  Russell A. Epstein,et al.  Scene Perception in the Human Brain. , 2019, Annual review of vision science.

[23]  Amy M. Belfi,et al.  The default-mode network represents aesthetic appeal that generalizes across visual domains , 2019, Proceedings of the National Academy of Sciences.

[24]  The Oxford Handbook of Empirical Aesthetics , 2019 .

[25]  Le Wu,et al.  Aesthetic Attributes Assessment of Images , 2019, ACM Multimedia.

[26]  Radoslaw Martin Cichy,et al.  Object Vision in a Structured World , 2019, Trends in Cognitive Sciences.

[27]  H. T. Schie,et al.  The neural correlates of the awe experience: Reduced default mode network activity during feelings of awe , 2019, Human brain mapping.

[28]  Radoslaw Martin Cichy,et al.  Deep Neural Networks as Scientific Models , 2019, Trends in Cognitive Sciences.

[29]  Natasha Z. Kirkham,et al.  Gorilla in our midst: An online behavioral experiment builder , 2018, Behavior Research Methods.

[30]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Li Fei-Fei,et al.  Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior , 2018, eLife.

[32]  J. Wagemans,et al.  Beauty in the blink of an eye: The time course of aesthetic experiences , 2018, British journal of psychology.

[33]  Marius V. Peelen,et al.  Transformation from independent to integrative coding of multi-object arrangements in human visual cortex , 2017, NeuroImage.

[34]  Yaoda Xu,et al.  Task-context-dependent Linear Representation of Multiple Visual Objects in Human Parietal Cortex , 2017, Journal of Cognitive Neuroscience.

[35]  Tom Hartley,et al.  Patterns of response to scrambled scenes reveal the importance of visual properties in the organization of scene-selective cortex , 2017, Cortex.

[36]  Tobias Preis,et al.  Using deep learning to quantify the beauty of outdoor places , 2017, Royal Society Open Science.

[37]  Benedek Kurdi,et al.  Introducing the Open Affective Standardized Image Set (OASIS) , 2017, Behavior research methods.

[38]  Christopher Baldassano,et al.  Human‐Object Interactions Are More than the Sum of Their Parts , 2016, Cerebral cortex.

[39]  James V. Haxby,et al.  CoSMoMVPA: Multi-Modal Multivariate Pattern Analysis of Neuroimaging Data in Matlab/GNU Octave , 2016, bioRxiv.

[40]  Tom Hartley,et al.  Patterns of neural response in scene-selective regions of the human brain are affected by low-level manipulations of spatial frequency , 2016, NeuroImage.

[41]  J. Wagemans,et al.  Brain-decoding fMRI reveals how wholes relate to the sum of parts , 2015, Cortex.

[42]  Nikolaus Kriegeskorte,et al.  Deep neural networks: a new framework for modelling biological vision and brain information processing , 2015, bioRxiv.

[43]  Liang Wang,et al.  Probabilistic Maps of Visual Topography in Human Cortex. , 2015, Cerebral cortex.

[44]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[45]  James Zijun Wang,et al.  RAPID: Rating Pictorial Aesthetics using Deep Learning , 2014, ACM Multimedia.

[46]  Marius V Peelen,et al.  Object grouping based on real-world regularities facilitates perception by reducing competitive interactions in visual cortex , 2014, Proceedings of the National Academy of Sciences.

[47]  R. Tootell,et al.  Thinking Outside the Box: Rectilinear Shapes Selectively Activate Scene-Selective Cortex , 2014, The Journal of Neuroscience.

[48]  Katharina N. Seidl,et al.  Whole person-evoked fMRI activity patterns in human fusiform gyrus are accurately modeled by a linear combination of face- and body-evoked activity patterns. , 2014, Journal of neurophysiology.

[49]  Dwight J. Kravitz,et al.  Deconstructing visual scenes in cortex: gradients of object and spatial layout information. , 2013, Cerebral cortex.

[50]  Karen B. Schloss,et al.  Visual aesthetics and human preference. , 2013, Annual review of psychology.

[51]  Nancy Kanwisher,et al.  An algorithmic method for functionally defining regions of interest in the ventral visual pathway , 2012, NeuroImage.

[52]  Karen B. Schloss,et al.  Aesthetic response to color combinations: preference, harmony, and similarity , 2010, Attention, perception & psychophysics.

[53]  Sabine Kastner,et al.  Defining the Units of Competition: Influences of Perceptual Organization on Competitive Interactions in Human Visual Cortex , 2010, Journal of Cognitive Neuroscience.

[54]  Dirk B. Walther,et al.  Natural Scene Categories Revealed in Distributed Patterns of Activity in the Human Brain , 2009, The Journal of Neuroscience.

[55]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Russell A. Epstein,et al.  Decoding the Representation of Multiple Simultaneous Objects in Human Occipitotemporal Cortex , 2009, Current Biology.

[57]  Paul J. Silvia,et al.  Do People Prefer Curved Objects? Angularity, Expertise, and Aesthetic Preference , 2009 .

[58]  Daniel M. Oppenheimer.  The secret life of fluency , 2008, Trends in Cognitive Sciences.

[59]  S. Kastner,et al.  Stimulus similarity modulates competitive interactions in human visual cortex. , 2007, Journal of vision.

[60]  Edward A. Vessel,et al.  The neural basis of scene preferences , 2007, Neuroreport.

[61]  N. Schwarz,et al.  Processing Fluency and Aesthetic Pleasure: Is Beauty in the Perceiver's Processing Experience? , 2004, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[62]  Magnus Enquist,et al.  Symmetry, beauty and evolution , 1994, Nature.