Role of Awareness and Universal Context in a Spiking Conscious Neural Network (SCNN): A New Perspective and Future Directions

Awareness plays a major role in human cognition and adaptive behaviour, though mechanisms involved remain unknown. Awareness is not an objectively established fact, therefore, despite extensive research, scientists have not been able to fully interpret its contribution in multisensory integration and precise neural firing, hence, questions remain: (1) How the biological neuron integrates the incoming multisensory signals with respect to different situations? (2) How are the roles of incoming multisensory signals defined (selective amplification/attenuation) that help neuron(s) to originate a precise neural firing complying with the anticipated behavioural-constraint of the environment? (3) How are the external environment and anticipated behaviour integrated? Recently, scientists have exploited deep learning to integrate multimodal cues and capture context-dependent meanings. Yet, these methods suffer from imprecise behavioural representation. In this research, we introduce a new theory on the role of awareness and universal context that can help answering the aforementioned crucial neuroscience questions. Specifically, we propose a class of spiking conscious neuron in which the output depends on three functionally distinctive integrated input variables: receptive field (RF), local contextual field (LCF), and universal contextual field (UCF). The RF defines the incoming ambiguous sensory signal, LCF defines the modulatory signal coming from other parts of the brain, and UCF defines the awareness. It is believed that the conscious neuron inherently contains enough knowledge about the situation in which the problem is to be solved based on past learning and reasoning and it defines the precise role of incoming multisensory signals to originate a precise neural firing (exhibiting switch-like behaviour). It is shown that the conscious neuron helps modelling a more precise human behaviour.

[1]  Jon Barker,et al.  An audio-visual corpus for speech perception and automatic speech recognition. , 2006, The Journal of the Acoustical Society of America.

[2]  Q Summerfield,et al.  Use of Visual Information for Phonetic Perception , 1979, Phonetica.

[3]  Amir Hussain,et al.  Contextual Audio-Visual Switching For Speech Enhancement in Real-World Environments , 2018, Inf. Fusion.

[4]  Amir Hussain,et al.  A novel brain-inspired compression-based optimised multimodal fusion for emotion recognition , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[5]  Kevin Wilson,et al.  Looking to listen at the cocktail party , 2018, ACM Trans. Graph..

[6]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[7]  Jon Barker,et al.  The third ‘CHiME’ speech separation and recognition challenge: Dataset, task and baselines , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[8]  Jianpei Zhang,et al.  Microblog sentiment analysis using social and topic context , 2018, PloS one.

[9]  Jim Kay,et al.  Contrasting information theoretic decompositions of modulatory and arithmetic interactions in neural information processing systems , 2018, ArXiv.

[10]  Jim W Kay,et al.  Coherent Infomax as a Computational Goal for Neural Systems , 2011, Bulletin of mathematical biology.

[11]  J. Werker,et al.  Two-month-old infants match phonetic information in lips and voice , 2003 .

[12]  E. Gelenbe G-networks by triggered customer movement , 1993 .

[13]  Stefan Feuerriegel,et al.  Decision support from financial disclosures with deep neural networks and transfer learning , 2017, Decis. Support Syst..

[14]  T. Stanford,et al.  The neural basis of multisensory integration in the midbrain: Its organization and maturation , 2009, Hearing Research.

[15]  Jon Barker,et al.  DNN driven Speaker Independent Audio-Visual Mask Estimation for Speech Separation , 2018, INTERSPEECH.

[16]  Amir Hussain,et al.  Deep learning driven multimodal fusion for automated deception detection , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[17]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[18]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[19]  Brian S. Stensrud,et al.  Formalizing context-based reasoning: A modeling paradigm for representing tactical human behavior , 2008 .

[20]  Yu Tsao,et al.  Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networks , 2017, IEEE Transactions on Emerging Topics in Computational Intelligence.

[21]  Amir Hussain,et al.  Lip-Reading Driven Deep Learning Approach for Speech Enhancement , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[22]  Dario Floreano,et al.  Contextually guided unsupervised learning using local multivariate binary processors , 1998, Neural Networks.

[23]  Tobias Bonhoeffer,et al.  Selective Persistence of Sensorimotor Mismatch Signals in Visual Cortex of Behaving Alzheimer’s Disease Mice , 2016, Current Biology.