Improving Mandarin Tone Recognition Based on DNN by Combining Acoustic and Articulatory Features Using Extended Recognition Networks

The tone is a distinctive feature in Mandarin Chinese. Tone recognition is useful in distinguishing ambiguous words in Chinese Mandarin speech recognition. Most traditional studies focused on prosodic features (e.g., F0, duration and energy) to improve the performance of tone recognition. In this paper, we propose a novel framework to integrate articulatory features (AFs) and MFCC into a DNN-HMM based tone recognition system. The procedure was implemented as the following steps: 1) estimating posterior probabilities of different AFs using a DNN classifier; 2) combining the estimated posterior probabilities with MFCC and F0 as input features; 3) realizing tone recognition using DNN-HMM. F0 and Energy were used as the input features of the baseline system based on DNN-HMM. Three contrast experiments were conducted according to different features under the framework of DNN: MFCC only, MFCC+F0 and MFCC+F0+AFs. The experimental results showed that the system with MFCC+F0 features outperformed the baseline system, with a 46.8% relative error reduction. After incorporating the AFs, a further relative reduction of about 10.1% in the tone error rate was achieved, which showed the efficiency of the proposed method.