The automatic analysis of classroom talk

The SMART SPEECH Project is a joint venture between three Finnish universities and a Chilean university. The aim is to develop a mobile application that can be used to record classroom talk and enable observations to be made of classroom interactions. We recorded Finnish and Chilean physics teachers’ speech using both a conventional microphone/dictator setup and a microphone/mobile application setup. The recordings were analysed via automatic speech recognition (ASR). The average word error rate achieved for the Finnish teachers’ speech was under 40%. The ASR approach also enabled us to determine the key topics discussed within the Finnish physics lessons under scrutiny. The results here were promising as the recognition accuracy rate was about 85% on average.

[1]  Ian Abrahams,et al.  Does Practical Work Really Work? A study of the effectiveness of practical work as a teaching and learning method in school science , 2008 .

[2]  Krzysztof Marasek,et al.  SPEECON – Speech Databases for Consumer Devices: Database Specification and Validation , 2002, LREC.

[3]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[4]  Steve Young,et al.  Token passing: a simple conceptual model for connected speech recognition systems , 1989 .

[5]  Lalit R. Bahl,et al.  Decoding for channels with insertions, deletions, and substitutions with applications to speech recognition , 1975, IEEE Trans. Inf. Theory.

[6]  Yiming Wang,et al.  Purely Sequence-Trained Neural Networks for ASR Based on Lattice-Free MMI , 2016, INTERSPEECH.

[7]  Biing-Hwang Juang,et al.  Maximum likelihood estimation for multivariate mixture observations of markov chains , 1986, IEEE Trans. Inf. Theory.

[8]  P. Scott,et al.  Constructing Scientific Knowledge in the Classroom , 1994 .

[9]  P. Scott,et al.  Individual and Sociocultural Views of Learning in Science Education , 2003 .

[10]  J. Paul Robinson,et al.  Using speech recognition for real-time captioning and lecture transcription in the classroom , 2013, IEEE Transactions on Learning Technologies.

[11]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[12]  Daniel Povey,et al.  The Kaldi Speech Recognition Toolkit , 2011 .

[13]  Andrew Olney,et al.  A Study of Automatic Speech Recognition in Noisy Classroom Environments for Automated Dialog Analysis , 2015, AIED.

[14]  Karen Spärck Jones.  A statistical interpretation of term specificity and its application in retrieval , 2021, J. Documentation.

[15]  Yasuo Ariki,et al.  Topic segmentation and retrieval system for lecture videos based on spontaneous speech recognition , 2003, INTERSPEECH.

[16]  Xingyu Pan,et al.  Automatic classification of activities in classroom discourse , 2014, Comput. Educ..

[17]  T. Seidel,et al.  Effects of a classroom discourse intervention on teachers' practice and students' motivation to learn mathematics and science , 2015 .

[18]  Tara N. Sainath,et al.  FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597 , 2012 .

[19]  Hervé Bourlard,et al.  Detecting speaker roles and topic changes in multiparty conversations using latent topic models , 2014, INTERSPEECH.

[20]  Mikko Kurimo,et al.  Studies on Training Text Selection for Conversational Finnish Language Modeling , 2013 .

[21]  Mikko Kurimo,et al.  ASR in Classroom Today: Automatic Visualization of Conceptual Network in Science Classrooms , 2017, EC-TEL.