Embodied Interactions with E-Textiles and the Internet of Sounds for Performing Arts

This paper presents initial steps towards the design of an embedded system for body-centric sonic performance. The proposed prototyping system allows performers to manipulate sounds through gestural interactions captured by textile wearable sensors. The e-textile sensor data control, in real-time, audio synthesis algorithms working with content from Audio Commons, a novel web-based ecosystem for re-purposing crowd-sourced audio. The system enables creative embodied music interactions by combining seamless physical e-textiles with web-based digital audio technologies.

[1]  Victor Lazzarini.  Erratum: New Digital Musical Instruments: Control and Interaction Beyond the Keyboard , 2008, Computer Music Journal.

[2]  Qi Wang.  Designing Posture Monitoring Garments to Support Rehabilitation , 2016, Tangible and Embedded Interaction.

[3]  Advait Jain,et al.  Tactile sensing over articulated joints with stretchable sensors , 2013, 2013 World Haptics Conference (WHC).

[4]  Xavier Serra,et al.  Querying Freesound with a microphone , 2015 .

[5]  Carlo Fischione,et al.  Towards the Internet of Musical Things , 2017 .

[6]  Patrick G. T. Healey,et al.  Talking Through Your Arse: Sensing Conversation with Seat Covers , 2017, CogSci.

[7]  Lucy E. Dunne,et al.  Garment-based body sensing using foam sensors , 2006, AUIC.

[8]  S Puckette Miller,et al.  Pure Data : another integrated computer music environment , 1996 .

[9]  Andrew P. McPherson,et al.  An Environment for Submillisecond-Latency Audio and Sensor Processing on BeagleBone Black , 2015 .

[10]  Camille Baker,et al.  Hacking the Body 2.0: Flutter/Stutter , 2016 .

[11]  Norbert Schnell,et al.  Sound Selection by Gestures , 2011, NIME.

[12]  Sara Price,et al.  Towards an integrated methodological framework for understanding embodiment in HCI , 2014, CHI Extended Abstracts.

[13]  Allison Druin,et al.  Technology probes: inspiring design for and with families , 2003, CHI '03.

[14]  Xavier Serra,et al.  Freesound technical demo , 2013, ACM Multimedia.

[15]  Mark D. Plumbley,et al.  Audio Commons: bringing Creative Commons audio content to the creative industries , 2016 .

[16]  James McCartney,et al.  Rethinking the Computer Music Language: SuperCollider , 2002, Computer Music Journal.

[17]  Lida Xu,et al.  The internet of things: a survey , 2014, Information Systems Frontiers.

[18]  Mel Alexenberg.  The Future of Art in a Postdigital Age: From Hellenistic to Hebraic Consciousness - Second Edition , 2011 .

[19]  Neil M. White,et al.  Fabric-based Strain Sensors for Measuring Movement in Wearable Telemonitoring Applications , 2009 .

[20]  Mark D. Plumbley,et al.  The Serendiptichord: Reflections on the Collaborative Design Process between Artist and Researcher , 2013, Leonardo.

[21]  Brian Christopher Smith,et al.  Query by humming: musical information retrieval in an audio database , 1995, MULTIMEDIA '95.

[22]  Anna Xambó.  Embodied Music Interaction: Creative Design Synergies Between Music Performance and HCI , 2017 .

[23]  Ivan Poupyrev,et al.  Augmented Groove: Collaborative Jamming in Augmented Reality , 2000, SIGGRAPH 2000.

[24]  Carlo Fischione,et al.  Smart instruments : Towards an ecosystem of interoperable devices connecting performers and audiences , 2019 .