Musical Smart City: Perspectives on Ubiquitous Sonification

Smart cities are urban areas with sensor networks that collect data used towards efficient management. As a source of ubiquitous data, smart city initiatives present opportunities to enhance inhabitants' urban awareness. However, making sense of smart city data is challenging and there is a gap between available data and end-user applications. Sonification emerges as a promising method for the interpretation of smart city data and the production of novel musical experiences. In this paper, we first present the smart city paradigm. We then cover the topics of ubiquitous and mobile music, followed by an overview of sonification research. Finally, we propose an approach entitled ubiquitous sonification and present the initial design of a speculative use case for musical smart city systems, leveraging user and urban data to inform behaviour.

[1]  Terri L. Bonebright,et al.  Sonific ation Report: Status of the Field and Research Agenda , 2010 .

[2]  Mathieu Barthet,et al.  Datascaping: Data Sonification as a Narrative Device in Soundscape Composition , 2017, Audio Mostly Conference.

[3]  Thomas Hermann,et al.  Introduction to the special issue on interactive sonification , 2019, Journal on Multimodal User Interfaces.

[4]  Kirsty A. Beilharz,et al.  An Interface and Framework Design for Interactive Aesthetic Sonification , 2009 .

[5]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[6]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[7]  Ge Wang,et al.  MoMu: A Mobile Music Toolkit , 2010, NIME.

[8]  Sang Won Lee,et al.  Mobile Devices as Musical Instruments - State of the Art and Future Prospects , 2017, CMMR.

[9]  Thomas Hermann,et al.  The Discipline of Interactive Sonification , 2004 .

[10]  Lars Erik Holmquist.  Ubiquitous music , 2005, INTR.

[11]  Gary E. Burnett,et al.  Sculpting a Mobile Musical Soundtrack , 2015, CHI.

[12]  Atau Tanaka,et al.  Mobile Music Technology: Report on an Emerging Community , 2006, NIME.

[13]  Alexandra Supper,et al.  The Search for the “Killer Application”: Drawing the Boundaries around the Sonification of Scientific Data , 2011 .

[15]  Oded Nov,et al.  SONYC: A System for the Monitoring, Analysis and Mitigation of Urban Noise Pollution , 2018, ArXiv.

[16]  Nuria Bonet Filella.  Data sonification in creative practice , 2019 .

[17]  Jonas Löwgren,et al.  SONIFYING THE PERIPHERY: SUPPORTING THE FORMATION OF GESTALT IN AIR TRAFFIC CONTROL , 2016 .

[18]  Carlo Fischione,et al.  Internet of Musical Things: Vision and Challenges , 2018, IEEE Access.

[19]  Leonidas Anthopoulos,et al.  Understanding Smart Cities: A Tool for Smart Government or an Industrial Trick? , 2017 .

[20]  Alan B. Craig,et al.  Using sound to extract meaning from complex data , 1991, Electronic Imaging.

[21]  Thomas Hermann,et al.  TAXONOMY AND DEFINITIONS FOR SONIFICATION AND AUDITORY DISPLAY , 2008 .

[22]  Brandt Dainow.  Smart City Transcendent: Understanding the smart city by transcending ontology , 2017 .

[23]  Michael A. Nees,et al.  Auditory Graphs Are Not the “Killer App” of Sonification, But They Work , 2018, Ergonomics in Design: The Quarterly of Human Factors Applications.

[24]  Erin V. L. Vasudevan,et al.  Interactive Sonification of Gait: Realtime BioFeedback for People with Parkinson’s Disease , 2016 .

[25]  Stephen Barrass.  The aesthetic turn in sonification towards a social and cultural medium , 2011, AI & SOCIETY.

[26]  T. Stockman,et al.  Towards a Method , Techniques and Tools to Support the Development of Auditory Displays , 2014 .

[27]  Federico Cugurullo.  The origin of the smart city imaginary , 2018, The Routledge Companion to Urban Imaginaries.

[28]  John G. Neuhoff,et al.  Sonification Report: Status of the Field and Research Agenda Prepared for the National Science Foundation by members of the International Community for Auditory Display , 1999 .

[29]  S. Rendón.  Understanding media : the extensions of man , 1967 .

[30]  Roberto Bresin,et al.  A Systematic Review of Mapping Strategies for the Sonification of Physical Quantities , 2013, PloS one.

[31]  Francisco Ramos,et al.  Promoting Pollution-Free Routes in Smart Cities Using Air Quality Sensor Networks , 2018, Sensors.

[32]  Lars Erik Holmquist,et al.  Sonic City: The Urban Environment as a Musical Interface , 2003, NIME.

[33]  Catherine Guastavino,et al.  Sounds in the city: bridging the gaps from research to practice through soundscape workshops , 2020 .

[34]  U. Berardi,et al.  Smart Cities: Definitions, Dimensions, Performance, and Initiatives , 2015 .

[35]  A. Supper.  Lobbying for the ear : the public fascination with and academic legitimacy of the sonification of scientific data , 2012 .

[36]  Stephen Barrass.  Sonic Information Design , 2015, ICAD.

[37]  Leonidas G. Anthopoulos,et al.  The Rise of the Smart City , 2017 .

[38]  Bruce N. Walker,et al.  Hearing Artificial Intelligence: Sonification Guidelines & Results From a Case-study in Melanoma Diagnosis , 2019 .

[39]  Mahadev Satyanarayanan,et al.  Pervasive computing: vision and challenges , 2001, IEEE Wirel. Commun..

[40]  Songnian Li,et al.  Geospatial Dashboards for Monitoring Smart City Performance , 2019, Sustainability.

[41]  Davide Rocchesso,et al.  The Sonification Handbook , 2011 .

[42]  John G. Neuhoff,et al.  Is Sonification Doomed to Fail? , 2019, Proceedings of the 25th International Conference on Auditory Display (ICAD 2019).

[43]  Rebecca Fiebrink,et al.  Cross-modal Sound Mapping Using Deep Learning , 2013, NIME.