An ubiquitous smart guitar system for collaborative musical practice

ABSTRACT Ubiquitous music (UbiMus) proposes to study how social interaction with mobile and distributed technologies can converge to form novel creativity support tools and music artistic practices. A recent field overlapping with UbiMus is the Internet of Musical Things, which refers to ecosystems of interconnected embedded computers (Musical Things) enabling users to produce, interact with or experience musical content. Musical Things embed electronics, sensors, data forwarding and processing software into physical or virtual objects. Smart musical instruments (SMIs) are an emerging class of Musical Things provided with capabilities of capturing and receiving data supporting instrumental musical practice. Due to their portability and self-containdeness, SMIs enable novel ubiquitous interactions between performers of acoustic and digital musical instruments. After a review of current trends in SMI research, we propose an ubiquitous smart guitar system which uses the guitar as a hub for collaborative music making. We then present a survey conducted with 18 performers to assess the usability, creativity support and engagement with the system. Results show a positive emotional engagement with the system which overall was found easy to use and novel. We also discuss several barriers to creative interaction related to the size of the user interface, creative agency and personalisation.

[1]  Mark Weiser.  The computer for the 21st century , 1991 .

[2]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[3]  Mahadev Satyanarayanan,et al.  Pervasive computing: vision and challenges , 2001, IEEE Wirel. Commun..

[4]  Malcolm Slaney,et al.  Semantic-audio retrieval , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Ning Hu,et al.  Pattern Discovery Techniques for Music Audio , 2002, ISMIR.

[6]  Matthew Wright,et al.  Problems and Prospects for Intimate Musical Control of Computers , 2002, Computer Music Journal.

[7]  J. Stephen Downie,et al.  Music information retrieval , 2005, Annu. Rev. Inf. Sci. Technol..

[8]  J. Fereday,et al.  Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development , 2006 .

[9]  Marcelo M. Wanderley,et al.  New Digital Musical Instruments: Control And Interaction Beyond the Keyboard (Computer Music and Digital Audio Series) , 2006 .

[10]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[11]  Edgar Berdahl,et al.  Advancements in Actuated Musical Instruments , 2011, Organised Sound.

[12]  Nick Bryan-Kinns,et al.  Identifying mutual engagement , 2012, Behav. Inf. Technol..

[13]  Xavier Serra,et al.  Freesound technical demo , 2013, ACM Multimedia.

[14]  Eleonora Borgia,et al.  The Internet of Things vision: Key features, applications and open issues , 2014, Comput. Commun..

[15]  Daniel Müllensiefen,et al.  The Musicality of Non-Musicians: An Index for Assessing Musical Sophistication in the General Population , 2014, PloS one.

[16]  Marcelo Soares Pimenta,et al.  Ubiquitous Music , 2014, Computational Music Science.

[17]  Edgar Berdahl,et al.  How to Make Embedded Acoustic Instruments , 2014, NIME.

[18]  Andrew R. Nix,et al.  Making the Most of Wi-Fi: Optimisations for Robust Wireless Live Music Performance , 2014, NIME.

[19]  Celine Latulipe,et al.  Quantifying the Creativity Support of Digital Tools through the Creativity Support Index , 2014, ACM Trans. Comput. Hum. Interact..

[20]  P. Driessen,et al.  Reconfigurable Autonomous Novel Guitar Effects ( RANGE ) , 2014 .

[21]  Andrew P. McPherson,et al.  A Low-Cost Real-Time Tracking System for Violin , 2015 .

[22]  Andrew P. McPherson,et al.  An Environment for Submillisecond-Latency Audio and Sensor Processing on BeagleBone Black , 2015 .

[23]  Susanne Bødker,et al.  Third-wave HCI, 10 years later---participation and sharing , 2015, Interactions.

[24]  Ann Light,et al.  Designing Connected Products: UX for the Consumer Internet of Things , 2015 .

[25]  Andrew P. McPherson.  Buttons, Handles, and Keys: Advances in Continuous-Control Keyboard Instruments , 2015, Computer Music Journal.

[26]  Marcelo Soares Pimenta,et al.  Prototyping of Ubiquitous Music Ecosystems , 2015, J. Cases Inf. Technol..

[27]  Baptiste Caramiaux,et al.  The Machine Learning Algorithm as Creative Musical Tool , 2016, ArXiv.

[28]  Andrew P. McPherson,et al.  Action-Sound Latency: Are Our Tools Fast Enough? , 2016, NIME.

[29]  Augusto Sarti,et al.  An Overview on Networked Music Performance Technologies , 2016, IEEE Access.

[30]  Mark D. Plumbley,et al.  Audio Commons: bringing Creative Commons audio content to the creative industries , 2016 .

[31]  Yongmeng Wu,et al.  Open Symphony: Creative Participation for Audiences of Live Music Performances , 2017, IEEE Multim..

[32]  Victor Lazzarini,et al.  Ecologically Grounded Creative Practices in Ubiquitous Music , 2017, Organised Sound.

[33]  Koray Tahiroglu.  Mobile music making paradigm: towards a new culture of use , 2017 .

[34]  G. Fitzpatrick,et al.  Design Implications for Technology-Mediated Audience Participation in Live Music , 2017 .

[35]  Carlo Fischione,et al.  Examples of use cases with Smart Instruments , 2017, Audio Mostly Conference.

[36]  Luca Turchet,et al.  Smart Mandolin: autobiographical design, implementation, use cases, and lessons learned , 2018, Audio Mostly Conference.

[37]  Carlo Fischione,et al.  Musicians' initial encounters with a smart guitar , 2018, NordiCHI.

[38]  Andrew R. Brown,et al.  How Ubiquitous Technologies Support Ubiquitous Music , 2018 .

[39]  Luca Turchet,et al.  Real-Time Hit Classification in a Smart Cajón , 2018, Front. ICT.

[40]  Luca Turchet,et al.  Some reflections on the relation between augmented and smart musical instruments , 2018, Audio Mostly Conference.

[41]  Damián Keller,et al.  Theoretical Approaches to Musical Creativity: The Ubimus Perspective , 2018 .

[42]  Luca Turchet,et al.  Jamming with a Smart Mandolin and Freesound-based Accompaniment , 2018, 2018 23rd Conference of Open Innovations Association (FRUCT).

[43]  Andrew P. McPherson,et al.  Co-design of a Smart Cajón , 2018 .

[44]  Carlo Fischione,et al.  Internet of Musical Things: Vision and Challenges , 2018, IEEE Access.

[45]  Mathieu Barthet,et al.  Playsound.space: Inclusive Free Music Improvisations Using Audio Commons , 2018, NIME.

[46]  Steve Benford,et al.  Encumbered Interaction: a Study of Musicians Preparing to Perform , 2019, CHI.

[47]  Luca Turchet,et al.  Smart Musical Instruments: Vision, Design Principles, and Future Directions , 2019, IEEE Access.

[48]  Luca Turchet,et al.  Co-Design of Musical Haptic Wearables for Electronic Music Performer's Communication , 2019, IEEE Transactions on Human-Machine Systems.

[49]  Damián Keller,et al.  The Handy Metaphor: Bimanual, touchless interaction for the internet of musical things , 2019, Journal of New Music Research.