Interactive sonification and the IoT: the case of smart sonic shoes for clinical applications

To date, little attention has been devoted by the research community to applications of the Internet of Things (IoT) paradigm to the field of interactive sonification. The IoT has the potential to facilitate the emergence of novel forms of interactive sonifications that are the result of shared control of the sonification system by both the user performing the gestures locally to the system itself, and one or more remote users. This can for instance impact therapies based on auditory feedback where the control of the sound generation may be shared by patients and doctors remotely connected. This paper describes a prototype of connected shoes for interactive sonification that can be remotely controlled and can collect data about the gait of a walker. The system targets primarily clinical applications where sound stimuli are utilized to help guide and improve walking actions of patients with motor impairments.

[1]  Oded Nov,et al.  SONYC , 2019, Commun. ACM.

[2]  Luca Turchet,et al.  SoleSound: Towards a novel portable system for audio-tactile underfoot feedback , 2014, 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics.

[3]  Andrew P. McPherson,et al.  An Environment for Submillisecond-Latency Audio and Sensor Processing on BeagleBone Black , 2015 .

[4]  Davide Rocchesso,et al.  The Sonification Handbook , 2011 .

[5]  Luca Turchet,et al.  Footstep sounds synthesis: Design, implementation, and evaluation of foot–floor interactions, surface materials, shoe types, and walkers’ features , 2016 .

[6]  Murtaza Haider,et al.  Beyond the hype: Big data concepts, methods, and analytics , 2015, Int. J. Inf. Manag..

[7]  Matthew W. M. Rodger,et al.  Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson's Disease , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Luca Turchet,et al.  Walking pace affected by interactive sounds simulating stepping on different terrains , 2013, ACM Trans. Appl. Percept..

[9]  Lida Xu,et al.  The internet of things: a survey , 2014, Information Systems Frontiers.

[10]  Andy Hunt,et al.  Guest Editors' Introduction: An Introduction to Interactive Sonification , 2005, IEEE Multim..

[11]  Michael H. Thaut,et al.  Rhythmic Auditory Stimulation in Rehabilitation of Movement Disorders: A Review Of Current Research , 2010 .

[12]  Ana Tajadura-Jiménez,et al.  As Light as your Footsteps: Altering Walking Sounds to Change Perceived Body Weight, Emotional State and Gait , 2015, CHI.

[13]  Nick S. Jones,et al.  Robust, real-time and autonomous monitoring of ecosystems with an open, low-cost, networked device , 2017, bioRxiv.

[14]  VirtanenTuomas,et al.  Detection and Classification of Acoustic Scenes and Events , 2018 .

[15]  Carlo Fischione,et al.  Internet of Musical Things: Vision and Challenges , 2018, IEEE Access.

[16]  Luca Turchet,et al.  Embodied Interactions with E-Textiles and the Internet of Sounds for Performing Arts , 2018, TEI.

[17]  Peter E. Keller,et al.  Gait improvement via rhythmic stimulation in Parkinson’s disease is linked to rhythmic skills , 2017, Scientific Reports.

[18]  Luca Turchet,et al.  Smart Mandolin: autobiographical design, implementation, use cases, and lessons learned , 2018, Audio Mostly Conference.