An ensemble-based framework for mispronunciation detection of Arabic phonemes

Determinationofmispronunciationsandensuringfeedbacktousersaremaintainedbycomputer-assistedlanguagelearning(CALL)systems.Inthiswork,weintroduceanensemblemodelthat definesthemispronunciationofArabicphonemesandassistslearningofArabic,effectively.To the best of our knowledge, this is the very first attempt to determine the mispronunciations of Arabic phonemes employing ensemble learning techniques and conventional machine learning models, comprehensively. In order to observe the effect of feature extraction techniques, mel-frequency cepstrum coefficients (MFCC), and Mel spectrogram are blended with each learning algorithm. To show the success of proposed model, 29 letters in the Arabic phonemes, 8 of which are hafiz, are voiced by a total of 11 different person. The amount of data set has been enhanced employing the methods of adding noise, time shifting, time stretching, pitch shifting. Extensive experiment results demonstrate that the utilization of voting classifier as an ensemble algorithm with Mel spectrogram feature extraction technique exhibits remarkable classification result with 95.9% of accuracy.

[1]  S. Shareef,et al.  Comparison Between Features Extraction Techniques for Impairments Arabic Speech , 2022, Al-Rafidain Engineering Journal (AREJ).

[2]  M. Alsulaiman,et al.  Mispronunciation Detection and Diagnosis with Articulatory-Level Feedback Generation for Non-Native Arabic Speech , 2022, Mathematics.

[3]  Yanlu Xie,et al.  Text-Aware End-to-end Mispronunciation Detection and Diagnosis , 2022, ArXiv.

[4]  J. Hansen,et al.  Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment , 2022, INTERSPEECH.

[5]  H. F. Ahmad,et al.  An Approach for Pronunciation Classification of Classical Arabic Phonemes Using Deep Learning , 2021, Applied Sciences.

[6]  Javeria Farooq,et al.  Mispronunciation Detection in Articulation Points of Arabic Letters using Machine Learning , 2021, 2021 International Conference on Computing, Electronic and Electrical Engineering (ICE Cube).

[7]  Sungyoung Lee,et al.  ArCAR: A Novel Deep Learning Computer-Aided Recognition for Character-Level Arabic Text Representation and Recognition , 2021, Algorithms.

[8]  Muhammad Nadeem Majeed,et al.  An Arabic Mispronunciation Detection System Based on the Frequency of Mistakes for Asian Speakers , 2021, April 2021.

[9]  Muhammad Zia,et al.  Correct Pronunciation Detection of the Arabic Alphabet Using Deep Learning , 2021, Applied Sciences.

[10]  Yousaf Bin Zikria,et al.  Improving Mispronunciation Detection of Arabic Words for Non-Native Learners Using Deep Convolutional Neural Network Features , 2020 .

[11]  Muhammad Nadeem Majeed,et al.  Mispronunciation Detection Using Deep Convolutional Neural Network Features and Transfer Learning-Based Model for Arabic Phonemes , 2019, IEEE Access.

[12]  Patrick Cardinal,et al.  End-to-End Environmental Sound Classification using a 1D Convolutional Neural Network , 2019, Expert Syst. Appl..

[13]  Selçuk Alp,et al.  Makine Öğrenmesinde Sınıflandırma Yöntemleri ve R Uygulamaları , 2019 .

[14]  Eman K. Elsayed,et al.  Evaluation of quran recitation via OWL ontology based system , 2019, Int. Arab J. Inf. Technol..

[15]  A. Ewees,et al.  A Dataset for Speech Recognition to Support Arabic Phoneme Pronunciation , 2018 .

[16]  Kyunghyun Cho,et al.  A Comparison of Audio Signal Preprocessing Methods for Deep Neural Networks on Music Tagging , 2017, 2018 26th European Signal Processing Conference (EUSIPCO).

[17]  H. A. Habib,et al.  A Comparative Study of Classifier Based Mispronunciation Detection System for Confusing , 2017 .

[18]  Selim Akyokus,et al.  The effectiveness of homogenous ensemble classifiers for Turkish and English texts , 2016, 2016 International Symposium on INnovations in Intelligent SysTems and Applications (INISTA).

[19]  Ponnuthurai N. Suganthan,et al.  Ensemble Classification and Regression-Recent Developments, Applications and Future Directions [Review Article] , 2016, IEEE Computational Intelligence Magazine.

[20]  Muazzam Maqsood,et al.  A Complete Mispronunciation Detection System for Arabic Phonemes using SVM , 2016 .

[21]  Osama Abufanas,et al.  Computer Aided Language Learning System for Arabic for Second Language Learners , 2013 .

[22]  Ali Fauzi Ahmad Khan,et al.  Automatic Arabic pronunciation scoring for computer aided language learning , 2013, 2013 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA).

[23]  Namrata Dave,et al.  Feature Extraction Methods LPC, PLP and MFCC In Speech Recognition , 2013 .

[24]  Frank K. Soong,et al.  A new DNN-based high quality pronunciation evaluation for computer-aided language learning (CALL) , 2013, INTERSPEECH.

[25]  Bagus Tris Atmaja,et al.  Developing Speech Recognition System for Quranic Verse Recitation Learning Software , 2012 .

[26]  Airil Haimi Mohd Adnan,et al.  Computer-Aided Self-Access Language Learning: Views of Indonesian, Malaysian & New Zealand Practitioners , 2012 .

[27]  Ana María Martínez Enríquez,et al.  Voice Content Matching System for Quran Readers , 2010, 2010 Ninth Mexican International Conference on Artificial Intelligence.

[28]  Néstor Becerra Yoma,et al.  Automatic intonation assessment for computer aided language learning , 2010, Speech Commun..

[29]  Geoffrey I. Webb.  Naïve Bayes , 2010, Encyclopedia of Machine Learning.

[30]  Lior Rokach,et al.  Ensemble-based classifiers , 2010, Artificial Intelligence Review.

[31]  Thomas Seidl,et al.  k-Nearest Neighbor Classification , 2009, Encyclopedia of Database Systems.

[32]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[33]  Khaled Shaalan,et al.  An Intelligent Computer Assisted Language Learning System for Arabic Learners , 2005 .

[34]  Bernard Zenko,et al.  Is Combining Classifiers with Stacking Better than Selecting the Best One? , 2004, Machine Learning.

[35]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[36]  Nobuaki Minematsu,et al.  Pronunciation assessment based upon the compatibility between a learner's pronunciation structure and the target language's lexical structure , 2004, INTERSPEECH.

[37]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[38]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[39]  Mitch Weintraub,et al.  Automatic scoring of pronunciation quality , 2000, Speech Commun..

[40]  Lou Boves,et al.  Assessment of dutch pronunciation by means of automatic speech recognition technology , 1998, ICSLP.

[41]  Farzad Ehsani,et al.  Speech Technology in Computer-Assisted Language Learning: Strengths and Limitations of a New CALL Paradigm. , 1998 .

[42]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[43]  E. B. Newman,et al.  A Scale for the Measurement of the Psychological Magnitude Pitch , 1937 .