Tonal Language Processing

Atonal language uses changes in tone or pitch of a voiced sound to differentiate words. A classic example is the consonant-vowel combination /ma/ in Mandarin Chinese. The same /ma/, depending upon the tonal pattern of vowel /a/, can mean mother (妈, flat pattern), numb (麻, rising), horse (马, falling-rising), or curse (骂, falling). Growing up in the United States, my 9-year-old boy still confuses mother with horse, “cursing” his weekly 2-hour Chinese School as a form of “child abuse.” Who should we blame for inventing tonal languages? What’s good in them? Why is it hard for our brains? Or is it really? According to the late linguist, Yuen-Ren Chao, tones have been used to differentiate words in Chinese for at least 3,000 years. Recently researchers from the University of Edinburgh found that people who speak tonal languages also carry the least disturbed form of a 37,000-year-old gene, Microcephalin, suggesting that the first language was tonal (Dediu and Ladd, 2007). Indeed, ancient Greek (9-6th centuries BC) used tonal accents, but its tonality got lost, perhaps as a result of variations in that gene. Today, about 70% of the world’s languages are tonal languages, which are spoken by over 2-billion people, mostly in sub-Saharan Africa and South East Asia (Haviland et al., 2007). So, our ancestors out of Africa invented tonal languages, but why? One answer may lie in the acoustics and perception of tones. Dr. Zhi-An Liang at the Shanghai Institute of Physiology published a classical paper (Liang, 1963) to show that compared with consonant and vowel perception, tone perception is the most redundant in terms of resiliency to acoustic distortions. Although tones are defined by variations in fundamental frequency, they can still be accurately perceived after removing the fundamental frequency via highpass filtering or whispered speech. One can literally abuse the acoustic signal by filtering, infinite clipping, or adding noise, but still achieve a high level of tone perception. The reason for this high resistance to distortions and noise is that the acoustical cues for tone perception are multi-dimensional and widely distributed in both time and frequency domains. Tonal information is correlated with duration and temporal envelope in the time domain (Whalen and Xu, 1992; Fu et al., 1998). But the more salient cues for tone perception are in the temporal fine structure, fundamental frequency, and their harmonics (Xu and Pfingst, 2003; Kong and Zeng, 2006). Possibly for their acoustical redundancy and perceptual resiliency, tones were invented to enable long distance communication in noisy backgrounds. Well, they are still used today by Spanish-speaking villagers who can whistle Silbo in the Canary Islands (Meyer, 2008) as well as tonallanguage-speaking customers in a noisy Chinese restaurant (Lee, 2007; Luo et al., 2009). How do our ears and brain work together to process tonal information? Our ears are essentially filter banks that decompose sounds into different frequency regions. The filter bandwidth is narrow and relatively constant for center frequencies less than 2,000 Hz, but increases linearly for center frequencies above 2,000 Hz. In cases of a voiced sound, the fundamental frequency and its lower harmonics are likely separated into different filters, whereas the higher harmonics are likely combined into one filter. Tonal information is extracted from the output of these auditory filters. There are at least three types of cues for pitch extraction. First, the fundamental frequency itself conveys a salient pitch percept by producing a strong timing cue that occurs in the right place or apical part of the cochlea. Second, the lower harmonics can also produce a salient pitch percept by generating a distinctive temporal and spatial pattern along the cochlea, a well-known phenomenon called the missing fundamental. Third, the unresolved high harmonics can produce a strong timing cue that is phase-locked to the fundamental frequency, but in the wrong place or basal part of the cochlea. Functionally, this envelope-based timing cue cannot provide a salient pitch percept (Zeng, 2002; Oxenham et al., 2004). Recent physiological studies have shed light on the brain’s representation of pitch and its usage in tonal language processing. In marmoset monkeys, researchers found that neurons in a restricted low-frequency cortical region respond to both pure tones and their missing fundamental harmonic counterpart (Bendor and Wang, 2005). This cortical region has been mapped to Heschl’s Gyrus in humans. Interestingly, in a study teaching English-speaking subjects to learn Mandarin tones, Wong and colleagues (2008) found that subjects, who were less successful in learning, showed a smaller Heschl’s Gyrus volume on the left, but not on the right hemisphere, relative to learners who were successful. This finding leads to a general question on hemisphere specialization of tone perception: Which hemisphere do we use to process lexical tonal information? Hemisphere specialization has been known for a long time in that the left hemisphere is for speech whereas the right hemisphere is for music processing. Tones are represented by changes in pitch—a salient music quality, but they also carry lexical meaning—a salient speech feature. TONAL LANGUAGE PROCESSING

[1]  S. Nittrouer,et al.  The Effects of Bilateral Electric and Bimodal Electric—Acoustic Stimulation on Language Development , 2009, Trends in amplification.

[2]  Michael C F Tong,et al.  Lexical Tone Perception Ability of Profoundly Hearing-Impaired Children: Performance of Cochlear Implant and Hearing Aid Users , 2010, Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology.

[3]  Chao-Yang Lee,et al.  Does Horse Activate Mother? Processing Lexical Tone in Form Priming , 2007, Language and speech.

[4]  F. Zeng,et al.  Importance of tonal envelope cues in Chinese speech recognition. , 1998, The Journal of the Acoustical Society of America.

[5]  D H Whalen,et al.  Information for Mandarin Tones in the Amplitude Contour and in Brief Segments , 1990, Phonetica.

[6]  D. Bendor,et al.  The neuronal representation of pitch in primate auditory cortex , 2005, Nature.

[7]  Ning Zhou,et al.  Tone production of Mandarin Chinese speaking children with cochlear implants. , 2007, International journal of pediatric otorhinolaryngology.

[8]  Michael K. Qin,et al.  Effects of introducing unprocessed low-frequency information on the reception of envelope-vocoder processed speech. , 2006, The Journal of the Acoustical Society of America.

[9]  Fan-Gang Zeng,et al.  Speech and melody recognition in binaurally combined acoustic and electric hearing. , 2005, The Journal of the Acoustical Society of America.

[10]  Bryan E Pfingst,et al.  Relative importance of temporal envelope and fine structure in lexical-tone perception. , 2003, The Journal of the Acoustical Society of America.

[11]  Andrew J Oxenham,et al.  Correct tonotopic representation is necessary for complex pitch perception. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Fan-Gang Zeng,et al.  Temporal and spectral cues in Mandarin tone recognition. , 2006, The Journal of the Acoustical Society of America.

[13]  Patrick C M Wong,et al.  Volume of left Heschl's Gyrus and linguistic pitch learning. , 2008, Cerebral cortex.

[14]  Contribution of Spectral Cues to Mandarin Lexical Tone Recognition in Normal-Hearing and Hearing-Impaired Mandarin Chinese Speakers , 2010, Ear and hearing.

[15]  Fan-Gang Zeng,et al.  Opposite patterns of hemisphere dominance for early auditory processing of lexical tones and consonants , 2006, Proceedings of the National Academy of Sciences.

[16]  D. Ladd,et al.  Linguistic tone is related to the population frequency of the adaptive haplogroups of two brain size genes, ASPM and Microcephalin , 2007, Proceedings of the National Academy of Sciences.

[17]  Julien Meyer,et al.  Typology and acoustic strategies of whistled languages: Phonetic comparison and perceptual cues of whistled vowels , 2008, Journal of the International Phonetic Association.

[18]  Fan-Gang Zeng,et al.  Cochlear Implants: System Design, Integration, and Evaluation , 2008, IEEE Reviews in Biomedical Engineering.

[19]  Fan-Gang Zeng,et al.  Temporal pitch in electric hearing , 2002, Hearing Research.