Implementation and preliminary evaluation of ‘C-tone’: A novel algorithm to improve lexical tone recognition in Mandarin-speaking cochlear implant users

Objectives: Because of limited spectral resolution, Mandarin-speaking cochlear implant (CI) users have difficulty perceiving fundamental frequency (F0) cues that are important to lexical tone recognition. To improve Mandarin tone recognition in CI users, we implemented and evaluated a novel real-time algorithm (C-tone) to enhance the amplitude contour, which is strongly correlated with the F0 contour. Methods: The C-tone algorithm was implemented in clinical processors and evaluated in eight users of the Nurotron NSP-60 CI system. Subjects were given 2 weeks of experience with C-tone. Recognition of Chinese tones, monosyllables, and disyllables in quiet was measured with and without the C-tone algorithm. Subjective quality ratings were also obtained for C-tone. Results: After 2 weeks of experience with C-tone, there were small but significant improvements in recognition of lexical tones, monosyllables, and disyllables (P < 0.05 in all cases). Among lexical tones, the largest improvements were observed for Tone 3 (falling–rising) and the smallest for Tone 4 (falling). Improvements with C-tone were greater for disyllables than for monosyllables. Subjective quality ratings showed no strong preference for or against C-tone, except for perception of own voice, where C-tone was preferred. Discussion: The real-time C-tone algorithm provided small but significant improvements for speech performance in quiet with no change in sound quality. Pre-processing algorithms to reduce noise and better real-time F0 extraction would improve the benefits of C-tone in complex listening environments. Conclusions: Chinese CI users’ speech recognition in quiet can be significantly improved by modifying the amplitude contour to better resemble the F0 contour.

[1]  Monita Chatterjee,et al.  Processing F0 with cochlear implants: Modulation frequency discrimination and speech intonation recognition , 2008, Hearing Research.

[2]  Jan-Olof Svantesson,et al.  Tone 4 and Tone 3 Discrimination in Modern Standard Chinese , 1986 .

[3]  F. Zeng,et al.  Importance of tonal envelope cues in Chinese speech recognition. , 1998, The Journal of the Acoustical Society of America.

[4]  Ning Zhou,et al.  Musical pitch and lexical tone perception with cochlear implants , 2011, International journal of audiology.

[5]  William M. Rabinowitz,et al.  Better speech recognition with cochlear implants , 1991, Nature.

[6]  Ning Zhou,et al.  Lexical Tone Perception with HiResolution and HiResolution 120 Sound-Processing Strategies in Pediatric Mandarin-Speaking Cochlear Implant Users , 2009, Ear and hearing.

[7]  R V Shannon,et al.  Effects of amplitude nonlinearity on phoneme recognition by cochlear implant users and normal-hearing listeners. , 1998, The Journal of the Acoustical Society of America.

[8]  Jan Wouters,et al.  Speech perception with F0mod, a cochlear implant pitch coding strategy , 2015, International journal of audiology.

[9]  Xin Luo,et al.  Concurrent-vowel and tone recognition by Mandarin-speaking cochlear implant users , 2009, Hearing Research.

[10]  Qian-Jie Fu,et al.  Melodic Contour Identification and Music Perception by Cochlear Implant Users , 2009, Annals of the New York Academy of Sciences.

[11]  Marc Moonen,et al.  Improved Music Perception with Explicit Pitch Coding in Cochlear Implants , 2006, Audiology and Neurotology.

[12]  B. Qi,et al.  Music and lexical tone perception in chinese adult cochlear implant users , 2012, The Laryngoscope.

[13]  A. Abramson TONAL EXPERIMENTS WITH WHISPERED THAI , 1972 .

[14]  Bryan E Pfingst,et al.  Features of stimulation affecting tonal-speech perception: implications for cochlear prostheses. , 2002, The Journal of the Acoustical Society of America.

[15]  D H Whalen,et al.  Information for Mandarin Tones in the Amplitude Contour and in Brief Segments , 1990, Phonetica.

[16]  Jan Wouters,et al.  Perception of Mandarin Chinese with cochlear implants using enhanced temporal pitch cues , 2012, Hearing Research.

[17]  Saturnino Maldonado-Bascón,et al.  Low-complexity F0-based speech/nonspeech discrimination approach for digital hearing aids , 2011, Multimedia Tools and Applications.

[18]  F. Zeng,et al.  Objective and subjective evaluations of the Nurotron Venus cochlear implant system via animal experiments and clinical trials , 2016, Acta oto-laryngologica.

[19]  Peggy B Nelson,et al.  Understanding speech in modulated interference: cochlear implant users and normal-hearing listeners. , 2003, The Journal of the Acoustical Society of America.

[20]  Jan Wouters,et al.  Better place-coding of the fundamental frequency in cochlear implants. , 2004, The Journal of the Acoustical Society of America.

[21]  Qian-Jie Fu,et al.  The number of spectral channels required for speech recognition depends on the difficulty of the listening situation. , 2004, Acta oto-laryngologica. Supplementum.

[22]  Stuart Rosen,et al.  Enhancement of temporal periodicity cues in cochlear implants: effects on prosodic perception and vowel identification. , 2005, The Journal of the Acoustical Society of America.

[23]  Xin Luo,et al.  Speech Recognition and Temporal Amplitude Modulation Processing by Mandarin-Speaking Cochlear Implant Users , 2008, Ear and hearing.

[24]  Pierre A. Hallé,et al.  Tone production in modern standard chinese : an electromyographic investigation , 1986 .

[25]  L. F. Willems,et al.  Measurement of pitch in speech: an implementation of Goldstein's theory of pitch perception. , 1982, The Journal of the Acoustical Society of America.

[26]  Yitao Mao,et al.  Lexical tone recognition in noise in normal-hearing children and prelingually deafened children with cochlear implants , 2017, International journal of audiology.

[27]  Zachary M. Smith,et al.  Chimaeric sounds reveal dichotomies in auditory perception , 2002, Nature.

[28]  Hugh J. McDermott,et al.  The Design and Evaluation of a Hearing Aid with Trainable Amplification Parameters , 2007, Ear and hearing.

[29]  Astrid van Wieringen,et al.  Improved fundamental frequency coding in cochlear implant signal processing. , 2009, The Journal of the Acoustical Society of America.

[30]  R V Shannon,et al.  Speech Recognition with Primarily Temporal Cues , 1995, Science.

[31]  Ning Zhou,et al.  Tone perception and production in pediatric cochlear implants users , 2011, Acta oto-laryngologica.

[32]  Andrew E. Vandali,et al.  Development of a temporal fundamental frequency coding strategy for cochlear implants. , 2011, The Journal of the Acoustical Society of America.

[33]  Qian-Jie Fu,et al.  Melodic Contour Identification by Cochlear Implant Listeners , 2007, Ear and hearing.

[34]  Peggy B Nelson,et al.  Factors affecting speech understanding in gated interference: cochlear implant users and normal-hearing listeners. , 2004, The Journal of the Acoustical Society of America.

[35]  Qian-Jie Fu,et al.  Noise Susceptibility of Cochlear Implant Users: The Role of Spectral Resolution and Smearing , 2005, Journal of the Association for Research in Otolaryngology.

[36]  F. Zeng,et al.  Identification of temporal envelope cues in Chinese tone recognition , 2000 .

[37]  Xin Luo,et al.  Concurrent-vowel and tone recognitions in acoustic and simulated electric hearing. , 2009, The Journal of the Acoustical Society of America.

[38]  J. Galvin,et al.  Validation of list equivalency for Mandarin speech materials to use with cochlear implant listeners , 2017, International journal of audiology.

[39]  Shiming Yang,et al.  Development and evaluation of the Nurotron 26-electrode cochlear implant system , 2015, Hearing Research.

[40]  Xin Luo,et al.  Enhancing Chinese tone recognition by manipulating amplitude envelope: implications for cochlear implants. , 2004, The Journal of the Acoustical Society of America.

[41]  J. E. Lieberman,et al.  Frequency limitations of the two-point central difference differentiation algorithm , 1982, Biological Cybernetics.

[42]  Andrew E Vandali,et al.  Enhancement of temporal cues to pitch in cochlear implants: effects on pitch ranking. , 2012, The Journal of the Acoustical Society of America.

[43]  R. Shannon,et al.  Speech recognition in noise as a function of the number of spectral channels: comparison of acoustic hearing and cochlear implants. , 2001, The Journal of the Acoustical Society of America.

[44]  Clemens Zierhofer,et al.  Temporal fine structure in cochlear implants: Preliminary speech perception results in Cantonese-speaking implant users , 2010, Acta oto-laryngologica.

[45]  Qian-Jie Fu,et al.  Effects of Speech Processing Strategy on Chinese Tone Recognition by Nucleus-24 Cochlear Implant Users , 2004, Ear and hearing.

[46]  Qian-Jie Fu,et al.  Melodic Pitch Perception and Lexical Tone Perception in Mandarin-Speaking Cochlear Implant Users , 2015, Ear and hearing.