Semantics Based on the Physical Characteristics of Facial Expressions Used to Produce Japanese Vowels

Previous studies have reported that verbal sounds are associated—non-arbitrarily—with specific meanings (e.g., sound symbolism and onomatopoeia), including visual forms of information such as facial expressions; however, it remains unclear how mouth shapes used to utter each vowel create our semantic impressions. We asked 81 Japanese participants to evaluate mouth shapes associated with five Japanese vowels by using 10 five-item semantic differential scales. The results reveal that the physical characteristics of the facial expressions (mouth shapes) induced specific evaluations. For example, the mouth shape made to voice the vowel “a” was the one with the biggest, widest, and highest facial components compared to other mouth shapes, and people perceived words containing that vowel sound as bigger. The mouth shapes used to pronounce the vowel “i” were perceived as more likable than the other four vowels. These findings indicate that the mouth shapes producing vowels imply specific meanings. Our study provides clues about the meaning of verbal sounds and what the facial expressions in communication represent to the perceiver.

[1]  T. Kambara,et al.  The Relationships Between Initial Consonants in Japanese Sound Symbolic Words and Familiarity, Multi-Sensory Imageability, Emotional Valence, and Arousal , 2021, Journal of Psycholinguistic Research.

[2]  Morten H. Christiansen,et al.  Affective Arousal Links Sound to Meaning , 2020, Psychological science.

[3]  Shigeto Kawahara,et al.  Sound symbolism and theoretical phonology , 2020, Lang. Linguistics Compass.

[4]  C. Spence,et al.  Tasting names: Systematic investigations of taste-speech sounds associations , 2020 .

[5]  A. Jacobs,et al.  Affective iconic words benefit from additional sound–meaning integration in the left amygdala , 2019, Human brain mapping.

[6]  Aleix M. Martinez,et al.  Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements , 2019, Psychological science in the public interest : a journal of the American Psychological Society.

[7]  Nahyun Kwon,et al.  Semantic Associations Dominate Over Perceptual Associations in Vowel–Size Iconicity , 2019, i-Perception.

[8]  Daniel N. Jones,et al.  Arbitrary signals of trustworthiness – social judgments may rely on facial expressions even with experimentally manipulated valence , 2019, Heliyon.

[9]  Lukas D Lopez,et al.  The Influence of Context on Distinct Facial Expressions of Disgust , 2019, Emotion.

[10]  A. J. Fridlund,et al.  Inside-Out: From Basic Emotions Theory to the Behavioral Ecology View , 2019, Journal of Nonverbal Behavior.

[11]  C. Pelachaud,et al.  Naturalistic Emotion Decoding From Facial Action Sets , 2019, Front. Psychol..

[12]  F. Fang,et al.  Perceptual learning and recognition confusion reveal the underlying relationships among the six basic emotions , 2018, Cognition & emotion.

[13]  A. Jacobs,et al.  Why 'piss' is ruder than 'pee'? The role of sound in affective meaning making , 2018, PloS one.

[14]  J. Adelman,et al.  Emotional sound symbolism: Languages rapidly signal valence via phonemes , 2018, Cognition.

[15]  A. Jacobs,et al.  Affective Congruence between Sound and Meaning of Words Facilitates Semantic Decision , 2018, Behavioral sciences.

[16]  A. Jacobs,et al.  The Sound of Words Evokes Affective Brain Responses , 2018, Brain sciences.

[17]  Makoto Miyatani,et al.  Spontaneous Facial Expressions Are Different from Posed Facial Expressions: Morphological Properties and Dynamic Sequences , 2017 .

[18]  Andrea Scarantino,et al.  How to Do Things with Emotional Expressions: The Theory of Affective Pragmatics , 2017 .

[19]  Shushi Namba,et al.  Spontaneous Facial Actions Map onto Emotional Experiences in a Non-social Context: Toward a Component-Based Approach , 2017, Front. Psychol..

[20]  S. Kita,et al.  Sound symbolism scaffolds language development in preverbal infants , 2015, Cortex.

[21]  S. Kita,et al.  The sound symbolism bootstrapping hypothesis for language acquisition and language evolution , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[22]  Martine Grice,et al.  Mood is linked to vowel type: the role of articulatory movements. , 2014, Emotion.

[23]  Y. Mitsukura,et al.  Effects of Music on Image Impression and Relationship between Impression and Physical Properties , 2013 .

[24]  Svetlana K. Perović,et al.  Visual Perception of Public Open Spaces in Niksic , 2012 .

[25]  N. Osaka Ideomotor response and the neural representation of implied crying in the human brain: An fMRI study using onomatopoeia1 , 2011 .

[26]  Mutsumi Imai,et al.  Japanese Sound-Symbolism Facilitates Word Learning in English-Speaking Children , 2011, Cogn. Sci..

[27]  Kazuko Shinohara,et al.  A Cross-linguistic Study of Sound Symbolism: The Images of Size , 2010 .

[28]  Eva Krumhuber,et al.  Effects of Dynamic Attributes of Smiles in Human and Synthetic Faces: A Simulated Job Interview Setting , 2009 .

[29]  S. Kita,et al.  Sound symbolism facilitates early verb learning , 2008, Cognition.

[30]  D. Maurer,et al.  The shape of boubas: sound-shape correspondences in toddlers and adults. , 2006, Developmental science.

[31]  W. Gaebel,et al.  Facial expressivity in the course of schizophrenia and depression , 2004, European Archives of Psychiatry and Clinical Neuroscience.

[32]  Kimihiro Suzuki,et al.  Brief Report: Useful Information for Face Perception Is Described with FACS , 2003 .

[33]  V. Ramachandran,et al.  Synaesthesia? A window into perception, thought and language , 2001 .

[34]  Fumio Hara,et al.  Dynamic control of lip-configuration of a mouth robot for Japanese vowels , 2000, Robotics Auton. Syst..

[35]  Shoko Hamano The sound-symbolic system of Japanese , 2000 .

[36]  Richard R. Klink Creating Brand Names With Meaning: The Use of Sound Symbolism , 2000 .

[37]  William Ickes,et al.  The development of meaning contexts for empathic accuracy: Channel and sequence effects. , 1999 .

[38]  Daniel N. McIntosh,et al.  Facial Movement, Breathing, Temperature, and Affect: Implications of the Vascular Theory of Emotional Efference , 1997 .

[39]  P. Ekman An argument for basic emotions , 1992 .

[40]  F. Strack,et al.  Inhibiting and facilitating conditions of the human smile: a nonobtrusive test of the facial feedback hypothesis. , 1988, Journal of personality and social psychology.

[41]  J. Ohala,et al.  An Ethological Perspective on Common Cross-Language Utilization of F₀ of Voice , 1984, Phonetica.

[42]  Shizuo Hiki,et al.  Characteristics of the mouth shape in the production of Japanese , 1982 .

[43]  S. Tomkins,et al.  What and Where are the Primary Affects? Some Evidence for a Theory , 1964, Perceptual and motor skills.

[44]  J. M. Kittross The measurement of meaning , 1959 .

[45]  E. Sapir A study in phonetic symbolism. , 1929 .

[46]  Alan J. Fridlund,et al.  The behavioral ecology view of facial displays, 25 years later , 2017 .

[47]  Rainer Reisenzein,et al.  Coherence between emotions and facial expressions: a research synthesis , 2017 .

[48]  R. Zajonc,et al.  Feeling and facial efference: implications of the vascular theory of emotion. , 1989, Psychological review.

[49]  Stanley S. Newman Further experiments in phonetic symbolism , 1933 .