Reference marking in children's computer-directed speech: an integrated analysis of discourse and gestures

Understanding the fine details of children’s speech and their gestural characteristics helps, among other things, in creating natural computer interfaces. We analyze reference marking in young children’s computer-directed speech using audio-video data from 3to 6-year-old children engaged in a series of age-appropriate computer tasks, using aWizard of Oz technique. Along with speech transcriptions and acoustic information, discourse (referential devices, conversational repairs) and gestural characteristics (hand/head movement type) were annotated in a synchronized multi-layer system. The results point to the developmental variability and the multimodal nature of the speech young children produce while interacting with the computer agent, suggesting that interfaces addressed to this age group should be specifically designed to integrate multisensory information as well as to adjust to the child’s specific needs and interactional style.