Geometric approaches in time-frequency to grade tones in Chinese as a foreign language students

Mandarin is a tonal language, meaning pitch is used to distinguish lexical meaning. Tone recognition occupies a significant role in learning Chinese. Automated grading metrics can provide language students with feedback in a non-threatening environment and without the high time and/or financial cost of individual feedback from a teacher. In this paper, we present some preliminary work on geometric measures in time-frequency to grade the quality of tone production in Chinese as a foreign language students. Our preliminary results show that the metrics can distinguish native from non-native speakers for all tones. The low computational cost of these metrics makes them especially suitable for mobile platforms.

[1]  Zhou Ning,et al.  Mandarin Chinese Tone Recognition with an Artificial Neural Network , 2006 .

[2]  Phil Rose,et al.  Considerations in the normalisation of the fundamental frequency of linguistic tone , 1987, Speech Commun..

[3]  Loren Lugosch,et al.  Tone Recognition Using Lifters and CTC , 2018, INTERSPEECH.

[4]  Keikichi Hirose,et al.  Tone nucleus modeling for Chinese lexical tone recognition , 2004, Speech Commun..

[5]  J. Flege.  The production of "new" and "similar" phones in a foreign language: evidence for the effect of equivalence classification , 1987 .

[6]  Mark Liberman,et al.  Production and Perception of Tone 3 Focus in Mandarin Chinese , 2016, Front. Psychol..

[7]  Xiaohu Yang,et al.  Aging Effect on Categorical Perception of Mandarin Tones 2 and 3 and Thresholds of Pitch Contour Discrimination. , 2017, American journal of audiology.

[8]  Dorothy M. Chun,et al.  Acquisition of L2 Mandarin Chinese tones with learner-created tone visualizations , 2015 .

[9]  Yaping Yang,et al.  ToneNet: A CNN Model of Tone Classification of Mandarin Chinese , 2019, INTERSPEECH.

[10]  Ren-Hua Wang,et al.  CDF-Matching for Automatic Tone Error Detection in Mandarin Call System , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[11]  Puming Zhan,et al.  Deep Learning Based Mandarin Accent Identification for Accent Robust ASR , 2019, INTERSPEECH.

[12]  Yen-Chen Hao,et al.  The Effects of Tone Training on Tone Perception Accuracy in Chinese Language Classrooms , 2019, Classroom Research on Chinese as a Second Language.

[13]  Zhuoming Chen,et al.  The categorical perception of Mandarin tones by children with speech development disorders , 2017 .

[14]  Allard Jongman,et al.  Identifying the distinctive acoustic cues of Mandarin tones , 2018, The Journal of the Acoustical Society of America.

[15]  Kukulska-hulmeAgnes,et al.  An overview of mobile assisted language learning , 2008 .

[16]  Yen-Chen Hao,et al.  Second language acquisition of Mandarin Chinese tones by tonal and non-tonal language speakers , 2012, J. Phonetics.

[17]  Keikichi Hirose,et al.  Tone recognition of Chinese continuous speech using tone critical segments , 1999, EUROSPEECH.

[18]  Wensheng Hou,et al.  Auditory Brainstem Representation of the Voice Pitch Contours in the Resolved and Unresolved Components of Mandarin Tones , 2018, Front. Neurosci..

[19]  Mangui Liang,et al.  Detecting tone errors in continuous Mandarin speech , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Stephanie Seneff,et al.  Towards Automatic Tone Correction in Non-native Mandarin , 2006, ISCSLP.

[21]  Ren-Hua Wang,et al.  Tone Evaluation of Chinese Continuous Speech Based on Prosodic Words , 2008, 2008 6th International Symposium on Chinese Spoken Language Processing.

[22]  Bo Xu,et al.  A preliminary exploration on tone error detection in Mandarin based on clustering , 2010, 2010 4th International Universal Communication Symposium.

[23]  C. Best.  A direct realist view of cross-language speech perception , 1995 .

[24]  Stephanie Seneff,et al.  Robust pitch tracking for prosodic modeling in telephone speech , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[25]  Stephanie Seneff,et al.  Annotation and features of non-native Mandarin tone quality , 2009, INTERSPEECH.

[26]  W. Hou,et al.  Temporal Coding of Voice Pitch Contours in Mandarin Tones , 2018, Front. Neural Circuits.

[27]  C. Best,et al.  Nonnative and second-language speech perception : commonalities and complementarities , 2007 .

[28]  Gina-Anne Levow,et al.  The functional load of tone in Mandarin is as high as that of vowels , 2004, Speech Prosody 2004.

[29]  Woei-Chyn Chu,et al.  Effects of emotional tones of voice on the acoustic and perceptual characteristics of Mandarin tones , 2018, The Journal of the Acoustical Society of America.

[30]  Frank K. Soong,et al.  Automatic Detection of Tone Mispronunciation in Mandarin , 2006, ISCSLP.

[31]  Chang Liu,et al.  Tone Classification in Mandarin Chinese Using Convolutional Neural Networks , 2016, INTERSPEECH.

[32]  Yi Xu.  Contextual tonal variations in Mandarin , 1997 .

[33]  Mike Levy,et al.  Computer-Assisted Language Learning: Context and Conceptualization , 1997 .