Active Listening and Expressive Communication for Children with Hearing Loss Using Getatable Environments for Creativity

Abstract This paper describes a system for accommodating active listening for persons with hearing aids or cochlear implants, with a special focus on children at an early stage of cognitive development and with additional physical disabilities. A system called the Soundscraper is proposed and consists of a software part in Pure data and a hardware part using an Arduino microcontroller with a combination of sensors. For both the software and hardware development it was important to always ensure that the system was flexible enough to cater for the very different conditions that are characteristic of the intended user group. The Soundscraper has been tested with 25 children with good results. An increased attention span was reported, as well as positively surprising reactions from children where the caregivers were unsure whether they could hear at all. The sound synthesis methods, the gesture sensors and the employed parameter mapping were all simple, but they provided a controllable and sufficiently complex sound environment even with limited interaction. A possible future outcome of the application is the adoption of long-time analysis of sound preferences as opposed to traditional audiological investigations.

[1]  Loïc Kessous,et al.  Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech , 2008, Affect and Emotion in Human-Computer Interaction.

[2]  Wendy L. Magee.  Electronic technologies in clinical music therapy: A survey of practice and attitudes , 2006 .

[3]  B. Wilson,et al.  Cochlear Implants: Principles & Practices , 2000 .

[4]  Kjetil Falkenberg Hansen,et al.  The Skipproof Virtual Turntable for High-Level Control of Scratching , 2010, Computer Music Journal.

[5]  Janice Light,et al.  Supporting the Communication, Language, and Literacy Development of Children with Complex Communication Needs: State of the Science and Future Research Priorities , 2012, Assistive technology : the official journal of RESNA.

[6]  Erika Skoe,et al.  Experience‐induced Malleability in Neural Encoding of Pitch, Timbre, and Timing , 2009, Annals of the New York Academy of Sciences.

[7]  J. Mueller,et al.  Comparison of Music Perception in Bilateral and Unilateral Cochlear Implant Users and Normal-Hearing Subjects , 2009, Audiology and Neurotology.

[8]  Oliver A. Johnson.  The Standard Definition , 1980 .

[9]  M. Runco,et al.  The Standard Definition of Creativity , 2012 .

[10]  U. Andersson,et al.  Challenges in designing virtual environments training social skills for children with autism , 2006 .

[11]  vahid rashed,et al.  international classification of functioning , 2012 .

[12]  K. Gfeller,et al.  Effects of training on timbre recognition and appraisal by postlingually deafened cochlear implant recipients. , 2002, Journal of the American Academy of Audiology.

[13]  Alessandro Scorpecci,et al.  Improving melody recognition in cochlear implant recipients through individualized frequency map fitting , 2010, European Archives of Oto-Rhino-Laryngology.

[14]  Anders Friberg.  pDM: An Expressive Sequencer with Real-Time Control of the KTH Music-Performance Rules , 2006 .

[15]  J. Meinzen-Derr,et al.  Perceived qualitative benefits of cochlear implants in children with multi-handicaps. , 2005, International journal of pediatric otorhinolaryngology.

[16]  S. Johnston,et al.  Children With Cochlear Implants , 2003 .

[17]  V. Looi,et al.  Music perception of cochlear implant users: A questionnaire, and its implications for a music training program , 2010, International journal of audiology.

[18]  Beng Hong. Goh,et al.  Effects of training. , 2001 .

[19]  Kjetil Falkenberg Hansen,et al.  Ljudskrapan/The Soundscraper : Sound exploration for children with complex needs, accommodating hearing aids and cochlear implants , 2011 .

[20]  A. Daneshi,et al.  Cochlear implantation in prelingually deaf persons with additional disability , 2006, The Journal of Laryngology & Otology.

[21]  Stephen N. Calculator,et al.  Augmentative and alternative communication (AAC) and inclusive education for students with the most severe disabilities , 2009 .

[22]  Eliezer Rapoport,et al.  Overtone Spectra of Gongs used in Music Therapy , 2008 .

[23]  Tom Chau,et al.  Movement-to-music computer technology: a developmental play experience for children with severe physical disabilities. , 2007, Occupational therapy international.

[24]  Music recognition by children with cochlear implants , 2004 .

[25]  L. Edwards.  Children with cochlear implants and complex needs: a review of outcome research and psychological practice. , 2007, Journal of deaf studies and deaf education.

[26]  Nicola Bernardini,et al.  Preface: Special Issue on the Future of Sound and Music Computing , 2007 .

[27]  Differences between electrode-assigned frequencies and cochlear implant recipient pitch perception , 2007, Acta oto-laryngologica.

[28]  V. Kudryavtsev.  The phenomenon of child creativity , 2011 .

[29]  Philipos C. Loizou,et al.  Mimicking the human ear , 1998, IEEE Signal Process. Mag..

[30]  Wendy L. Magee,et al.  Using Electronic Music Technologies in Music Therapy: Opportunities, Limitations and Clinical Indicators , 2008 .

[31]  C. Mindham.  Creativity and the young child , 2005 .

[32]  M. Müller-Wehlau.  Objective audiological diagnostics using novel acoustical and electrophysiological tests , 2011 .

[33]  Anders Friberg,et al.  pDM: An Expressive Sequencer with Real-Time Control of the KTH Music-Performance Rules , 2006, Computer Music Journal.

[34]  T. Nikolopoulos,et al.  Speech production in deaf implanted children with additional disabilities and comparison with age-equivalent implanted children without such disorders. , 2008, International journal of pediatric otorhinolaryngology.

[35]  M. Wass.  Children with cochlear implants : cognition and reading ability , 2009 .

[36]  Hugh J. McDermott.  Music Perception with Cochlear Implants: A Review , 2004, Trends in amplification.

[37]  T. Nikolopoulos,et al.  Diagnostic challenges and safety considerations in cochlear implantation under the age of 12 months. , 2010, International journal of pediatric otorhinolaryngology.

[38]  E. A. Darling.  The Effects of Training , 2010 .

[39]  J. Hanekom,et al.  Free field frequency discrimination abilities of cochlear implant users , 2008, Hearing Research.

[40]  Victor Lazzarini.  Erratum: New Digital Musical Instruments: Control and Interaction Beyond the Keyboard , 2008, Computer Music Journal.

[41]  J. Meinzen-Derr,et al.  Additional disabilities and communication mode in a pediatric cochlear implant population , 2004 .

[42]  R. Carlyon,et al.  Concurrent Sound Segregation in Electric and Acoustic Hearing , 2007, Journal of the Association for Research in Otolaryngology.

[43]  Diana Deutsch,et al.  Music perception. , 2007, Frontiers in Bioscience.

[44]  Takanori Hirose,et al.  Preface to special issue , 2014, Brain Tumor Pathology.