Automatized analysis of children’s exposure to child-directed speech in reschool settings: Validation and application

The present study explored whether a tool for automatic detection and recognition of interactions and child-directed speech (CDS) in preschool classrooms could be developed, validated, and applied to non-coded video recordings representing children’s classroom experiences. Using first-person video recordings collected by 13 preschool children during a morning in their classrooms, we extracted high-level audiovisual features from recordings using automatic speech recognition and computer vision services from a cloud computing provider. Using manual coding for interactions and transcriptions of CDS as reference, we trained and tested supervised classifiers and linear mappings to measure five variables of interest. We show that the supervised classifiers trained with speech activity, proximity, and high-level facial features achieve adequate accuracy in detecting interactions. Furthermore, in combination with an automatic speech recognition service, the supervised classifier achieved error rates for CDS measures that are in line with other open-source automatic decoding tools in early childhood settings. Finally, we demonstrate our tool’s applicability by using it to automatically code and transcribe children’s interactions and CDS exposure vertically within a classroom day (morning to afternoon) and horizontally over time (fall to winter). Developing and scaling tools for automatized capture of children’s interactions with others in the preschool classroom, as well as exposure to CDS, may revolutionize scientific efforts to identify precise mechanisms that foster young children’s language development.

[1]  J. Marc Overhage,et al.  A systematic comparison of contemporary automatic speech recognition engines for conversational clinical speech , 2018, AMIA.

[2]  J. Gilkerson,et al.  Assessing Children’s Home Language Environments Using Automatic Speech Recognition Technology , 2011 .

[3]  E. Hoff The specificity of environmental influence: socioeconomic status affects early vocabulary development via maternal speech. , 2003, Child development.

[4]  Franceska Xhakaj,et al.  EduSense: Practical Classroom Sensing at Scale , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[5]  Soroush Vosoughi,et al.  An Automatic Child-Directed Speech Detector for the Study of Child Language Development , 2012, INTERSPEECH.

[6]  Tara N. Sainath,et al.  Large vocabulary automatic speech recognition for children , 2015, INTERSPEECH.

[7]  L. L. Sperry,et al.  Reexamining the Verbal Environments of Children From Different Socioeconomic Backgrounds. , 2019, Child development.

[8]  Alejandrina Cristia,et al.  Accuracy of the Language Environment Analysis System Segmentation and Metrics: A Systematic Review. , 2020, Journal of speech, language, and hearing research : JSLHR.

[9]  J. Huttenlocher,et al.  Language input and child syntax , 2002, Cognitive Psychology.

[10]  L. Justice,et al.  Linguistic environment of preschool classrooms: What dimensions support children’s language growth? , 2018 .

[11]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[12]  Marc W. Hernandez,et al.  Enriching home language environment among families from low-SES backgrounds: A randomized controlled trial of a home visiting curriculum , 2020 .

[13]  Dwight W. Irvin,et al.  Exploring Classroom Behavioral Imaging: Moving Closer to Effective and Data-Based Early Childhood Inclusion Planning , 2017 .

[14]  M. Rowe,et al.  Language Matters: Denying the Existence of the 30-Million-Word Gap Has Serious Consequences. , 2018, Child development.

[15]  Udo Rudolph,et al.  Continuous measurement of dynamic classroom social interactions , 2019, International journal of behavioral development.

[16]  Anne Marie Tharpe,et al.  Remote Microphone System Use at Home: Impact on Child-Directed Speech. , 2019, Journal of speech, language, and hearing research : JSLHR.

[17]  Julia A. Leonard,et al.  Language Exposure Relates to Structural Neural Connectivity in Childhood , 2018, The Journal of Neuroscience.

[18]  Jessica A. R. Logan,et al.  Peer Effects in Early Childhood Education , 2014, Psychological science.

[19]  M. Rowe A longitudinal investigation of the role of quantity and quality of child-directed speech in vocabulary development. , 2012, Child development.

[20]  Florian Metze,et al.  The ACLEW DiViMe: An Easy-to-use Diarization Tool , 2018, INTERSPEECH.

[21]  Susan Goldin-Meadow,et al.  Language input and acquisition in a Mayan village: how important is directed speech? , 2012, Developmental science.

[22]  Jing Chen,et al.  The Social Networks of Children With and Without Disabilities in Early Childhood Special Education Classrooms , 2019, Journal of autism and developmental disorders.

[23]  D K Oller,et al.  Automated vocal analysis of naturalistic recordings from children with autism, language delay, and typical development , 2010, Proceedings of the National Academy of Sciences.

[24]  John H. L. Hansen,et al.  Signal processing for young child speech language development , 2008, WOCCI.

[25]  Jessica A. R. Logan,et al.  The preschool classroom linguistic environment: Children’s first-person experiences , 2019, PloS one.

[26]  I. Schwartz,et al.  A Comprehensive Peer Network Intervention to Improve Social Communication of Children with Autism Spectrum Disorders: A Randomized Trial in Kindergarten and First Grade , 2015, Journal of autism and developmental disorders.

[27]  D. Christakis,et al.  Assessment of a Parent-Child Interaction Intervention for Language Development in Children , 2019, JAMA network open.

[28]  Florian Metze,et al.  Automatic word count estimation from daylong child-centered recordings in various language environments using language-independent syllabification of speech , 2019, Speech Commun..

[29]  J. Ludbrook SPECIAL ARTICLE COMPARING METHODS OF MEASUREMENT , 1997 .

[30]  Rhodri Cusack,et al.  Using automatic face analysis to score infant behaviour from video collected online. , 2019, Infant behavior & development.

[31]  Jill Gilkerson,et al.  Transcriptional Analyses of the LENA Natural Language Corpus , 2009 .

[32]  David K. Dickinson,et al.  Patterns of Teacher–Child Conversations in Head Start Classrooms: Implications for an Empirically Grounded Approach to Professional Development , 2008 .

[33]  Shraddha Mane,et al.  Facial Recognition, Expression Recognition, and Gender Identification , 2018, Data Management, Analytics and Innovation.

[34]  Xiaofu He,et al.  Socioeconomic Disparities in Language Input Are Associated With Children's Language-Related Brain Structure and Reading Skills. , 2019, Child development.

[35]  Philippe Joly,et al.  Audiovisual diarization of people in video content , 2012, Multimedia Tools and Applications.

[36]  Alejandrina Cristia,et al.  A step-by-step guide to collecting and analyzing long-format speech environment (LFSE) recordings , 2019, Collabra: Psychology.

[37]  Shayne B. Piasta,et al.  Language- and literacy-learning opportunities in early childhood classrooms: Children's typical experiences and within-classroom variability , 2014 .

[38]  Brett Laursen,et al.  A year in words: The dynamics and consequences of language experiences in an intervention classroom , 2018, PloS one.

[39]  Dina C. Castro,et al.  The Case for Measuring and Reporting Bilingualism in Developmental Research , 2019, Collabra. Psychology.

[40]  Louise A. Kaczmarek,et al.  Interaction among preschoolers with and without disabilities: effects of across-the-day peer intervention. , 1997, Journal of speech, language, and hearing research : JSLHR.

[41]  M. Rowe,et al.  Child-directed speech: relation to socioeconomic status, knowledge of child development and child vocabulary skill* , 2008, Journal of Child Language.

[42]  R. Hashimoto,et al.  Sunk Cost Effect in Individuals with Autism Spectrum Disorder , 2018, Journal of autism and developmental disorders.

[43]  Andrew J. Mashburn,et al.  Quality of Language and Literacy Instruction in Preschool Classrooms Serving At-Risk Pupils. , 2008, Early childhood research quarterly.

[44]  J. Downer,et al.  Observations of Children's Interactions with Teachers, Peers, and Tasks across Preschool Classroom Activity Settings , 2012, Early education and development.