Consciousness is more than meets the eye: a call for a multisensory study of subjective experience†

Abstract Over the last 30 years, our understanding of the neurocognitive bases of consciousness has improved, mostly through studies employing vision. While studying consciousness in the visual modality presents clear advantages, we believe that a comprehensive scientific account of subjective experience must not neglect other exteroceptive and interoceptive signals as well as the role of multisensory interactions for perceptual and self-consciousness. Here, we briefly review four distinct lines of work which converge in documenting how multisensory signals are processed across several levels and contents of consciousness. Namely, how multisensory interactions occur when consciousness is prevented because of perceptual manipulations (i.e. subliminal stimuli) or because of low vigilance states (i.e. sleep, anesthesia), how interactions between exteroceptive and interoceptive signals give rise to bodily self-consciousness, and how multisensory signals are combined to form metacognitive judgments. By describing the interactions between multisensory signals at the perceptual, cognitive, and metacognitive levels, we illustrate how stepping out the visual comfort zone may help in deriving refined accounts of consciousness, and may allow cancelling out idiosyncrasies of each sense to delineate supramodal mechanisms involved during consciousness.

[1]  P. Haggard,et al.  Having a body versus moving your body: Neural signatures of agency and body-ownership , 2010, Neuropsychologia.

[2]  J. Giacino,et al.  The minimally conscious state: Definition and diagnostic criteria , 2002, Neurology.

[3]  A. Seth Interoceptive inference, emotion, and the embodied self , 2013, Trends in Cognitive Sciences.

[4]  O. Blanke,et al.  Balancing awareness: Vestibular signals modulate visual consciousness in the absence of awareness , 2015, Consciousness and Cognition.

[5]  H. Lau,et al.  How to measure metacognition , 2014, Front. Hum. Neurosci..

[6]  M. Piché,et al.  Age-related audiovisual interactions in the superior colliculus of the rat , 2016, Neuroscience.

[7]  C. Koch,et al.  Continuous flash suppression reduces negative afterimages , 2005, Nature Neuroscience.

[8]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[9]  Thomas Metzinger,et al.  Unconscious integration of multisensory bodily inputs in the peripersonal space shapes bodily self-consciousness , 2016, Cognition.

[10]  A. Revonsuo Binding and the Phenomenal Unity of Consciousness , 1999, Consciousness and Cognition.

[11]  H. Critchley,et al.  Multisensory integration across exteroceptive and interoceptive domains modulates self-experience in the rubber-hand illusion , 2013, Neuropsychologia.

[12]  S. Dehaene,et al.  Neural signature of the conscious processing of auditory regularities , 2009, Proceedings of the National Academy of Sciences.

[13]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[14]  N. Sobel,et al.  Humans can learn new information during sleep , 2012, Nature Neuroscience.

[15]  C. Guilleminault,et al.  Meta-analysis of quantitative sleep parameters from childhood to old age in healthy individuals: developing normative sleep values across the human lifespan. , 2004, Sleep.

[16]  M. Tsakiris,et al.  Heartfelt Self: Cardio‐Visual Integration Affects Self‐Face Recognition and Interoceptive Cortical Processing , 2016, Cerebral cortex.

[17]  O. Blanke,et al.  Transient Modulations of Neural Responses to Heartbeats Covary with Bodily Self-Consciousness , 2016, The Journal of Neuroscience.

[18]  A. Craig,et al.  How do you feel — now? The anterior insula and human awareness , 2009, Nature Reviews Neuroscience.

[19]  Maria V. Sanchez-Vives,et al.  Virtual Hand Illusion Induced by Visuomotor Correlations , 2010, PloS one.

[20]  C. Koch,et al.  Integrated information theory: from consciousness to its physical substrate , 2016, Nature Reviews Neuroscience.

[21]  Emmanuel Dupoux,et al.  Subliminal speech perception and auditory streaming , 2008, Cognition.

[22]  Justin A. Harris,et al.  Vision Merges With Touch in a Purely Tactile Discrimination , 2008, Psychological science.

[23]  R. Dolan,et al.  Metacognition: computation, biology and function , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[24]  M. Sigman,et al.  Auditory Feedback Differentially Modulates Behavioral and Neural Markers of Objective and Subjective Performance When Tapping to Your Heartbeat , 2015, Cerebral cortex.

[25]  O. Blanke,et al.  Posing for awareness: proprioception modulates access to visual consciousness in a continuous flash suppression task. , 2013, Journal of vision.

[26]  C. Rovee-Collier,et al.  Classical conditioning and retention of the infant's eyelid response: effects of age and interstimulus interval. , 1984, Journal of experimental child psychology.

[27]  Maria V. Sanchez-Vives,et al.  First Person Experience of Body Transfer in Virtual Reality , 2010, PloS one.

[28]  Ophelia Deroy,et al.  Metacognition in Multisensory Perception , 2016, Trends in Cognitive Sciences.

[29]  T. Nagel Brain bisection and the unity of consciousness , 1971, Synthese.

[30]  A. D. Weerd,et al.  The development of sleep during the first months of life , 2003 .

[31]  M. Morrone,et al.  Touch disambiguates rivalrous perception at early stages of visual analysis , 2010, Current Biology.

[32]  Randolph Blake,et al.  Psychophysical magic: rendering the visible ‘invisible’ , 2005, Trends in Cognitive Sciences.

[33]  A. Owen,et al.  Are There Levels of Consciousness? , 2016, Trends in Cognitive Sciences.

[34]  Natika Newton,et al.  Metacognition and consciousness , 1995 .

[35]  Mel Slater,et al.  Body ownership causes illusory self-attribution of speaking and influences subsequent real speaking , 2014, Proceedings of the National Academy of Sciences.

[36]  Pascal Mamassian,et al.  Confidence as a Common Currency between Vision and Audition , 2016, PloS one.

[37]  Dustin Stokes,et al.  Perception and Its Modalities , 2014 .

[38]  L. Populin Anesthetics Change the Excitation/Inhibition Balance That Governs Sensory Processing in the Cat Superior Colliculus , 2005, The Journal of Neuroscience.

[39]  Casey O'Callaghan,et al.  Grades of Multisensory Awareness , 2017 .

[40]  K Ikeda,et al.  Classical conditioning during human NREM sleep and response transfer to wakefulness. , 1996, Sleep.

[41]  Laurence R. Harris,et al.  Contralateral tactile masking between forearms , 2013, Experimental Brain Research.

[42]  Olaf Blanke,et al.  Turning Body and Self Inside Out , 2013, Psychological science.

[43]  C. Koch,et al.  Information integration without awareness , 2014, Trends in Cognitive Sciences.

[44]  Ophelia Deroy,et al.  Multisensory constraints on awareness , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[45]  H. Beh,et al.  Discrimination and Conditioning during Sleep as Indicated by the Electroencephalogram , 1965, Science.

[46]  J. Changeux,et al.  Experimental and Theoretical Approaches to Conscious Processing , 2011, Neuron.

[47]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[48]  Charles Spence,et al.  Is consciousness multisensory , 2013 .

[49]  E. Thompson,et al.  The Cambridge Handbook of Consciousness , 2007 .

[50]  Gideon Rothschild,et al.  Multisensory Integration of Natural Odors and Sounds in the Auditory Cortex , 2011, Neuron.

[51]  C Spence,et al.  The Complex Interplay Between Multisensory Integration and Perceptual Awareness. , 2016, Multisensory research.

[52]  David Alais,et al.  Touch Influences Visual Perception with a Tight Orientation-Tuning , 2013, PloS one.

[53]  Olaf Blanke,et al.  Behavioral, Neural, and Computational Principles of Bodily Self-Consciousness , 2015, Neuron.

[54]  D. Alais,et al.  Touch Interacts with Vision during Binocular Rivalry with a Tight Orientation Tuning , 2013, PloS one.

[55]  D. Crowell Studies in Early Infant Learning: Classical Conditioning of the Neonatal Heart Rate. , 1976 .

[56]  P. Balsam,et al.  Newborn infants learn during sleep , 2010, Proceedings of the National Academy of Sciences.

[57]  R. Stevenson,et al.  Detecting olfactory rivalry , 2013, Consciousness and Cognition.

[58]  O. Blanke,et al.  Behavioural, modeling, and electrophysiological evidence for domain-generality in human metacognition , 2016, bioRxiv.

[59]  M. Morrone,et al.  Early interaction between vision and touch during binocular rivalry. , 2013, Multisensory research.

[60]  N. Sobel,et al.  Human olfaction: a constant state of change-blindness , 2010, Experimental Brain Research.

[61]  C. Koch,et al.  Towards a neurobiological theory of consciousness , 1990 .

[62]  T. Metzinger,et al.  Video Ergo Sum: Manipulating Bodily Self-Consciousness , 2007, Science.

[63]  O. Blanke Multisensory brain mechanisms of bodily self-consciousness , 2012, Nature Reviews Neuroscience.

[64]  A. Gallace Multisensory Imagery , 2013, Springer New York.

[65]  Denise Chen,et al.  Olfaction Modulates Visual Perception in Binocular Rivalry , 2010, Current Biology.

[66]  D. Alais,et al.  Congruent tactile stimulation reduces the strength of visual suppression during binocular rivalry , 2015, Scientific Reports.

[67]  M. Sigman,et al.  Individual consistency in the accuracy and distribution of confidence judgments , 2016, Cognition.

[68]  Konrad Paul Kording,et al.  Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception , 2015, Front. Hum. Neurosci..

[69]  Jeroen J. A. van Boxtel,et al.  Introduction to research topic: attention and consciousness in different senses , 2013, Front. Psychol..

[70]  Marcia Grabowecky,et al.  Interactive Coding of Visual Spatial Frequency and Auditory Amplitude-Modulation Rate , 2012, Current Biology.

[71]  Emery N Brown,et al.  Dynamics of Propofol-Induced Loss of Consciousness Across Primate Neocortex , 2016, The Journal of Neuroscience.

[72]  B. Baars A Thoroughly Empirical Approach To Consciousness , 1994 .

[73]  C. Koch,et al.  Neural correlates of consciousness: progress and problems , 2016, Nature Reviews Neuroscience.

[74]  Uta Noppeney,et al.  A spatially collocated sound thrusts a flash into awareness , 2015, Front. Integr. Neurosci..

[75]  C. Clifford Binocular rivalry , 2009, Current Biology.

[76]  Philipp Sterzer,et al.  Neural processing of visual information under interocular suppression: a critical review , 2014, Front. Psychol..

[77]  Olaf Blanke,et al.  An invisible touch: Body-related multisensory conflicts modulate visual consciousness , 2016, Neuropsychologia.

[78]  B. Baars The conscious access hypothesis: origins and recent evidence , 2002, Trends in Cognitive Sciences.

[79]  W. Levelt On binocular rivalry , 1965 .

[80]  Olaf Blanke,et al.  Visual consciousness and bodily self-consciousness. , 2015, Current opinion in neurology.

[81]  Christof Koch,et al.  Multisensory Integration in Complete Unawareness , 2014, Psychological science.

[82]  O. Blanke,et al.  Changing motor perception by sensorimotor conflicts and body ownership , 2016, Scientific Reports.

[83]  M. Kleiner,et al.  Audiovisual interactions in binocular rivalry. , 2010, Journal of vision.

[84]  Asher Koriat,et al.  Metacognition and consciousness. , 2007 .

[85]  Steven Laureys The neural correlate of (un)awareness: lessons from the vegetative state , 2005, Trends in Cognitive Sciences.

[86]  S. Gallagher Philosophical conceptions of the self: implications for cognitive science , 2000, Trends in Cognitive Sciences.

[87]  C. Tallon-Baudry,et al.  Spontaneous fluctuations in neural responses to heartbeats predict visual detection , 2014, Nature Neuroscience.

[88]  C. Tallon-Baudry,et al.  The neural subjective frame: from bodily signals to perceptual consciousness , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[89]  Nathan Faivre,et al.  Invisible, but how? The depth of unconscious processing as inferred from different suppression techniques , 2014, Front. Psychol..

[90]  M. Morrone,et al.  Auditory and Tactile Signals Combine to Influence Vision during Binocular Rivalry , 2014, The Journal of Neuroscience.

[91]  A. Damasio The Feeling of What Happens: Body and Emotion in the Making of Consciousness , 1999 .

[92]  S. Porges,et al.  Heart Rate Conditioning in Newborn Infants: Relationships Among Conditionability, Heart Rate Variability, and Sex. , 1975 .

[93]  C. Koch,et al.  Some reflections on visual awareness. , 1990, Cold Spring Harbor symposia on quantitative biology.

[94]  A. de Weerd,et al.  The development of sleep during the first months of life. , 2003, Sleep medicine reviews.

[95]  H. Ehrsson The Experimental Induction of Out-of-Body Experiences , 2007, Science.

[96]  Kevin G. Munhall,et al.  Detection of Audiovisual Speech Correspondences Without Visual Awareness , 2013, Psychological science.

[97]  Rufin VanRullen,et al.  On the cyclic nature of perception in vision versus audition , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[98]  Maria V. Sanchez-Vives,et al.  From presence to consciousness through virtual reality , 2005, Nature Reviews Neuroscience.

[99]  C. Spence,et al.  Crossmodal Mental Imagery , 2013 .

[100]  A. Oxenham,et al.  Neural Correlates of Auditory Perceptual Awareness under Informational Masking , 2008, PLoS biology.