Neural correlates of text‐based emoticons: a preliminary fMRI study

Like nonverbal cues in oral interactions, text‐based emoticons, which are textual portrayals of a writer's facial expressions, are commonly used in electronic device–mediated communication. Little is known, however, about how text‐based emoticons are processed in the human brain. With this study, we investigated whether the text‐based emoticons are processed as face expressions using fMRI.

[1]  Daantje Derks,et al.  The role of emotion in computer-mediated communication: A review , 2008, Comput. Hum. Behav..

[2]  Bruno Rossion,et al.  Faces are represented holistically in the human occipito-temporal cortex , 2006, NeuroImage.

[3]  J. Dien.  A tale of two recognition systems: Implications of the fusiform face area and the visual word form area for lateralized object recognition models , 2009, Neuropsychologia.

[4]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[5]  Deborah A Yurgelun-Todd,et al.  Activation of the amygdala and anterior cingulate during nonconscious processing of sad versus happy faces , 2004, NeuroImage.

[6]  V. Menon,et al.  Saliency, switching, attention and control: a network model of insula function , 2010, Brain Structure and Function.

[7]  Israel Liberzon,et al.  Habituation of Rostral Anterior Cingulate Cortex to Repeated Emotionally Salient Pictures , 2003, Neuropsychopharmacology.

[8]  M. Tarr,et al.  The Fusiform Face Area is Part of a Network that Processes Faces at the Individual Level , 2000, Journal of Cognitive Neuroscience.

[9]  Bruno Rossion,et al.  Hemispheric Asymmetries for Whole-Based and Part-Based Face Processing in the Human Fusiform Gyrus , 2000, Journal of Cognitive Neuroscience.

[10]  George Bush,et al.  The emotional counting stroop paradigm: a functional magnetic resonance imaging probe of the anterior cingulate affective division , 1998, Biological Psychiatry.

[11]  Leslie G. Ungerleider,et al.  Visual Imagery of Famous Faces: Effects of Memory and Attention Revealed by fMRI , 2002, NeuroImage.

[12]  Y. Saalmann,et al.  Cognitive and Perceptual Functions of the Visual Thalamus , 2011, Neuron.

[13]  T. Allison,et al.  Temporal Cortex Activation in Humans Viewing Eye and Mouth Movements , 1998, The Journal of Neuroscience.

[14]  Naoki Mukawa,et al.  Brain activity when reading sentences and emoticons: an fMRI study of verbal and nonverbal communication , 2011 .

[15]  M. Buonocore,et al.  Posterior cingulate cortex activation by emotional words: fMRI evidence from a valence decision task , 2003, Human brain mapping.

[16]  N. Hadjikhani,et al.  Seeing Fearful Body Expressions Activates the Fusiform Cortex and Amygdala , 2003, Current Biology.

[17]  O. Churches,et al.  Emoticons in mind: An event-related potential study , 2014, Social neuroscience.

[18]  M. Posner,et al.  Cognitive and emotional influences in anterior cingulate cortex , 2000, Trends in Cognitive Sciences.

[19]  Ki Won Kwon,et al.  Objects and their icons in the brain: The neural correlates of visual concept formation , 2008, Neuroscience Letters.

[20]  Mark W. Woolrich,et al.  Bayesian analysis of neuroimaging data in FSL , 2009, NeuroImage.

[21]  N. Mukawa,et al.  Brain activity while reading sentences with kanji characters expressing emotions: an fMRI study on kanji characters expressing “laughter” and “tears” , 2011 .

[22]  Andrea Bergmann,et al.  Statistical Parametric Mapping The Analysis Of Functional Brain Images , 2016 .

[23]  C. Neuper,et al.  The functional correlates of face perception and recognition of emotional facial expressions as evidenced by fMRI , 2011, Brain Research.

[24]  J. Haxby,et al.  Distinct representations of eye gaze and identity in the distributed human neural system for face perception , 2000, Nature Neuroscience.

[25]  Jody C. Culham,et al.  The neural correlates of change detection in the face perception network , 2008, Neuropsychologia.

[26]  Lisa R. Betts,et al.  Decoding of Faces and Face Components in Face-Sensitive Human Visual Cortex , 2010, Front. Psychology.

[27]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[28]  Daantje Derks,et al.  Emoticons and Online Message Interpretation , 2008 .

[29]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[30]  N. Kanwisher,et al.  Stages of processing in face perception: an MEG study , 2002, Nature Neuroscience.

[31]  Joseph T. Devlin,et al.  Consistency and variability in functional localisers , 2009, NeuroImage.

[32]  N. Mukawa,et al.  Brain activity associated with graphic emoticons. The effect of abstract faces in communication over a computer network , 2011 .

[33]  Leslie G. Ungerleider,et al.  The functional organization of human extrastriate cortex: a PET-rCBF study of selective attention to faces and locations , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[34]  Mark W. Woolrich,et al.  FSL , 2012, NeuroImage.

[35]  Robert Lindenberg,et al.  Communication with emblematic gestures: Shared and distinct neural correlates of expression and reception , 2012, Human brain mapping.

[36]  Daantje Derks,et al.  Emoticons and social interaction on the Internet: the importance of social context , 2007, Comput. Hum. Behav..

[37]  Jia Liu,et al.  Perception of Face Parts and Face Configurations: An fMRI Study , 2010, Journal of Cognitive Neuroscience.

[38]  J. Devlin,et al.  Triple Dissociation of Faces, Bodies, and Objects in Extrastriate Cortex , 2009, Current Biology.

[39]  B. Vogt,et al.  Contributions of anterior cingulate cortex to behaviour. , 1995, Brain : a journal of neurology.

[40]  Peter Vuust,et al.  Amygdala and heart rate variability responses from listening to emotionally intense parts of a story , 2011, NeuroImage.

[41]  DAANTJE DERKS,et al.  Emoticons in Computer-Mediated Communication: Social Motives and Social Context , 2008, Cyberpsychology Behav. Soc. Netw..

[42]  D. Pitcher,et al.  The role of the occipital face area in the cortical face perception network , 2011, Experimental Brain Research.

[43]  R. Dolan,et al.  fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. , 2004, Journal of neurophysiology.

[44]  Qingyang Li,et al.  Emotional perception: Meta-analyses of face and natural scene processing , 2011, NeuroImage.

[45]  M. Hasselmo,et al.  The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey , 1989, Behavioural Brain Research.

[46]  Charles D. Smith,et al.  Neural substrates of facial emotion processing using fMRI. , 2001, Brain research. Cognitive brain research.

[47]  E. Bullmore,et al.  Investigation of facial recognition memory and happy and sad facial expression perception: an fMRI study , 1998, Psychiatry Research: Neuroimaging.

[48]  P. McGuire,et al.  Functional atlas of emotional faces processing: a voxel-based meta-analysis of 105 functional magnetic resonance imaging studies. , 2009, Journal of psychiatry & neuroscience : JPN.

[49]  Randolph G. Bias,et al.  How are icons processed by the brain? Neuroimaging measures of four types of visual stimuli used in information systems , 2015, J. Assoc. Inf. Sci. Technol..

[50]  Leslie G. Ungerleider,et al.  The Effect of Face Inversion on Activity in Human Neural Systems for Face and Object Perception , 1999, Neuron.

[51]  Shigeki Nakauchi,et al.  Facial color processing in the face‐selective regions: An fMRI study , 2014, Human brain mapping.

[52]  K. Luan Phan,et al.  Functional Neuroanatomy of Emotion: A Meta-Analysis of Emotion Activation Studies in PET and fMRI , 2002, NeuroImage.

[53]  Timothy J. Andrews,et al.  Neural responses to rigidly moving faces displaying shifts in social attention investigated with fMRI and MEG , 2010, Neuropsychologia.

[54]  Sabine Kastner,et al.  Functional imaging of the human lateral geniculate nucleus and pulvinar. , 2004, Journal of neurophysiology.

[55]  T. Allison,et al.  Face-Specific Processing in the Human Fusiform Gyrus , 1997, Journal of Cognitive Neuroscience.

[56]  K. Nakamura,et al.  Activation of the right inferior frontal cortex during assessment of facial emotion. , 1999, Journal of neurophysiology.

[57]  Morten Overgaard,et al.  Uncertainty and confidence from the triple-network perspective: Voxel-based meta-analyses , 2014, Brain and Cognition.

[58]  Elizabeth M. Brannon,et al.  The Neural Development of an Abstract Concept of Number , 2009, Journal of Cognitive Neuroscience.

[59]  N. Kanwisher,et al.  Mental Imagery of Faces and Places Activates Corresponding Stimulus-Specific Brain Regions , 2000, Journal of Cognitive Neuroscience.

[60]  Thomas T. Liu,et al.  The development of event-related fMRI designs , 2012, NeuroImage.

[61]  M. Seghier,et al.  A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. , 2003, Brain : a journal of neurology.

[62]  T. Allison,et al.  Differential Sensitivity of Human Visual Cortex to Faces, Letterstrings, and Textures: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.