A hybrid method for overlapping speech detection in classroom environment

Classroom discourse, which is a major composition of classroom, contains a slew of useful information such as providing the feedback to the teachers may lead to an improvement of the teaching quality. Generally, the classroom discourse can be divided into four categories/activities: teacher discourse, student discourse, quiet, and discussion. The automatic classification of activities provides a practical way to deal with the classroom discourse. However, the recognition of the discussion activity (a kind of overlapping speech) poses a great challenge to the treatment of the classroom discourse analysis. Therefore, in this paper, a new hybrid method based on the silence distribution and Independent Component Analysis (ICA) was proposed for the detection of overlapping speech. The results were found to be satisfactory and in good agreement with an experimental data. Based on these experimental results, the performance of the speaker segment of classroom event can be efficiently analyzed.

[1]  Tanja Schultz,et al.  Unsupervised Learning of Overlapped Speech Model Parameters For Multichannel Speech Activity Detection in Meetings , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[2]  Fabio Valente,et al.  Speaker diarization of overlapping speech based on silence distribution in meeting recordings , 2012, INTERSPEECH.

[3]  David A. van Leeuwen,et al.  Speech overlap detection in a two-pass speaker diarization system , 2009, INTERSPEECH.

[4]  Mari Ostendorf,et al.  Use of speaker location features in meeting diarization , 2008 .

[5]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[6]  Javier Hernando,et al.  The Detection of Overlapping Speech with Prosodic Features for Speaker Diarization , 2011, INTERSPEECH.

[7]  Naveed Sarfraz Khattak,et al.  Speaker Independent Urdu speech recognition using HMM , 2010, 2010 The 7th International Conference on Informatics and Systems (INFOS).

[8]  Jordi Luque,et al.  Simultaneous Speech Detection With Spatial Features for Speaker Diarization , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[9]  Andrew Olney,et al.  A Study of Automatic Speech Recognition in Noisy Classroom Environments for Automated Dialog Analysis , 2015, AIED.

[10]  Gerasimos Potamianos,et al.  Audio-visual speech activity detection in a two-speaker scenario incorporating depth information from a profile or frontal view , 2016, 2016 IEEE Spoken Language Technology Workshop (SLT).

[11]  Andreas Stolcke,et al.  Multispeaker speech activity detection for the ICSI meeting recorder , 2001, IEEE Workshop on Automatic Speech Recognition and Understanding, 2001. ASRU '01..

[12]  John H. L. Hansen,et al.  Teager–Kaiser Energy Operators for Overlapped Speech Detection , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[13]  Andreas Stolcke,et al.  Observations on overlap: findings and implications for automatic processing of multi-party conversation , 2001, INTERSPEECH.

[14]  Mary M. Juzwik.  Inspiring Dialogue: Talking to Learn in the English Classroom , 2013 .

[15]  Xingyu Pan,et al.  Automatic classification of activities in classroom discourse , 2014, Comput. Educ..

[16]  Yunlong Cai,et al.  Underdetermined blind separation of overlapped speech mixtures in time-frequency domain with estimated number of sources , 2017, Speech Commun..

[17]  Mieke Brekelmans,et al.  Real-time teacher-student interactions : a dynamic systems approach , 2014 .

[18]  Tsunenori Mine,et al.  Analysis of Students' Learning Activities through Quantifying Time-Series Comments , 2011, KES.

[19]  Dazhi Jiang,et al.  Emotional State Estimation from the Non-Contact, Automatic and Webcam-Based Human Cardiac Pulse , 2014 .

[20]  Carlos Segura,et al.  Overlap detection for speaker diarization by fusing spectral and spatial features , 2010, INTERSPEECH.

[21]  Helen F. Hastie,et al.  Information density and overlap in spoken dialogue , 2016, Comput. Speech Lang..

[22]  Hervé Bourlard,et al.  Overlapping Speech Detection Using Long-Term Conversational Features for Speaker Diarization in Meeting Room Conversations , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[23]  Subhash Basishtha,et al.  A study on HMM based speech recognition system , 2013, 2013 IEEE International Conference on Computational Intelligence and Computing Research.

[24]  Martin Zelenak.  Detection and Handling of Overlapping Speech for Speaker Diarization , 2012 .

[25]  Andrew Olney,et al.  Automatic Classification of Question & Answer Discourse Segments from Teacher's Speech in Classrooms. , 2015, EDM 2015.

[26]  Gerald Friedland,et al.  Improved Overlapped Speech Handling for Speaker Diarization , 2011, INTERSPEECH.