Perceptual Dominance in Brief Presentations of Mixed Images: Human Perception vs. Deep Neural Networks

Visual perception involves continuously choosing the most prominent inputs while suppressing others. Neuroscientists induce visual competitions in various ways to study why and how the brain makes choices of what to perceive. Recently deep neural networks (DNNs) have been used as models of the ventral stream of the visual system, due to similarities in both accuracy and hierarchy of feature representation. In this study we created non-dynamic visual competitions for humans by briefly presenting mixtures of two images. We then tested feed-forward DNNs with similar mixtures and examined their behavior. We found that both humans and DNNs tend to perceive only one image when presented with a mixture of two. We revealed image parameters which predict this perceptual dominance and compared their predictability for the two visual systems. Our findings can be used to both improve DNNs as models, as well as potentially improve their performance by imitating biological behaviors.

[1]  D H Foster,et al.  Human Sensitivity to Phase Perturbations in Natural Images: A Statistical Framework , 2000, Perception.

[2]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[3]  David L. Sheinberg,et al.  The role of temporal cortical areas in perceptual organization. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Thomas Martinetz,et al.  Deep convolutional neural networks as generic feature extractors , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[5]  Seyed-Mohsen Moosavi-Dezfooli,et al.  Universal Adversarial Perturbations , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[7]  N. Logothetis.  Single units and conscious vision. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[8]  David Alais,et al.  Monocular rivalry exhibits three hallmarks of binocular rivalry: Evidence for common processes , 2009, Vision Research.

[9]  Timothée Masquelier,et al.  Humans and Deep Networks Largely Agree on Which Kinds of Variation Make Object Recognition Harder , 2016, Front. Comput. Neurosci..

[10]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[11]  Kedarnath P Vilankar,et al.  Local masking in natural images: a database and analysis. , 2014, Journal of vision.

[12]  Alexei A. Efros,et al.  What makes ImageNet good for transfer learning? , 2016, ArXiv.

[13]  James J DiCarlo,et al.  Large-Scale, High-Resolution Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-of-the-Art Deep Artificial Neural Networks , 2018, The Journal of Neuroscience.

[14]  Benjamin Y. Hayden,et al.  Using a Simple Neural Network to Delineate Some Principles of Distributed Economic Choice , 2018, Front. Comput. Neurosci..

[15]  R. Blake,et al.  A fresh look at the temporal dynamics of binocular rivalry , 1989, Biological Cybernetics.

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Carson C. Chow,et al.  A Spiking Neuron Model for Binocular Rivalry , 2004, Journal of Computational Neuroscience.

[18]  C. Clifford.  Binocular rivalry , 2009, Current Biology.

[19]  Daniel L. K. Yamins,et al.  Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition , 2014, PLoS Comput. Biol..

[20]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[21]  N. Logothetis,et al.  Visual competition , 2002, Nature Reviews Neuroscience.

[22]  Nicholas J. Wade,et al.  On the Discovery of Monocular Rivalry by Tscherning in 1898: Translation and Review , 2017, i-Perception.

[23]  Marcel van Gerven,et al.  Convolutional neural network-based encoding and decoding of visual object recognition in space and time , 2017, NeuroImage.

[24]  Shimon Ullman,et al.  Atoms of recognition in human and computer vision , 2016, Proceedings of the National Academy of Sciences.

[25]  Tomaso Poggio,et al.  From Understanding Computation to Understanding Neural Circuitry , 1976 .

[26]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[27]  M. Hollins,et al.  The effect of contrast on the completeness of binocular rivalry suppression , 1980, Perception & psychophysics.

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Kenneth D. Miller,et al.  Understanding Biological Visual Attention Using Convolutional Neural Networks , 2017 .

[30]  W. Geisler,et al.  Retina-V1 model of detectability across the visual field. , 2014, Journal of vision.

[31]  Jonas Kubilius,et al.  Deep Neural Networks as a Computational Model for Human Shape Sensitivity , 2016, PLoS Comput. Biol..

[32]  P. Cavanagh,et al.  Onset Rivalry: Brief Presentation Isolates an Early Independent Phase of Perceptual Competition , 2007, PloS one.

[33]  D. Knill,et al.  Bayesian sampling in visual perception , 2011, Proceedings of the National Academy of Sciences.

[34]  Heiko H Schütt,et al.  An image-computable psychophysical spatial vision model. , 2017, Journal of vision.

[35]  David A. Leopold,et al.  What is rivalling during binocular rivalry? , 1996, Nature.

[36]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[37]  Dimitrios Pantazis,et al.  Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks , 2015, NeuroImage.

[38]  Grace W. Lindsay.  Feature-based Attention in Convolutional Neural Networks , 2015, ArXiv.

[39]  Noboru Ohnishi,et al.  A Neural Network Model of Dynamically Fluctuating Perception of Necker Cube as well as Dot Patterns , 1999, AAAI/IAAI.

[40]  Nikolaus Kriegeskorte,et al.  Deep neural networks: a new framework for modelling biological vision and brain information processing , 2015, bioRxiv.

[41]  Oliver Obst,et al.  The Müller-Lyer Illusion in a Computational Model of Biological Object Recognition , 2013, PloS one.

[42]  Matthias Bethge,et al.  Comparing deep neural networks against humans: object recognition when the signal gets weaker , 2017, ArXiv.

[43]  Hugh R Wilson,et al.  Computational evidence for a rivalry hierarchy in vision , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[44]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[45]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Marcel A. J. van Gerven,et al.  Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream , 2014, The Journal of Neuroscience.

[47]  J. Rinzel,et al.  Noise-induced alternations in an attractor network model of perceptual bistability. , 2007, Journal of neurophysiology.

[48]  D. Heeger,et al.  Neuronal activity in human primary visual cortex correlates with perception during binocular rivalry , 2000, Nature Neuroscience.

[49]  Nava Rubin,et al.  Balance between noise and adaptation in competition models of perceptual bistability , 2009, Journal of Computational Neuroscience.

[50]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[51]  R. Blake,et al.  Neural bases of binocular rivalry , 2006, Trends in Cognitive Sciences.