Measuring the Influences of Musical Parameters on Cognitive and Behavioral Responses to Audio Notifications Using EEG and Large-scale Online Studies

Prior studies have evaluated various designs for audio notifications. However, calls for more in-depth research on how such notifications work, especially at the level of users' cognitive states, have gone unanswered; and studies evaluating audio notifications with large numbers of participants in multiple environments have been rare. This study conducted an electroencephalography study (N=20) and an online study (N=967) to enhance understandings of how three musical parameters - melody (simple, complex), pitch (high, low), and tempo (fast, slow) - influenced users' cognition and behaviors. There are eight different notifications with different combinations of these parameters. The online study analyzed the effects of user-specific and environmental information on users' behaviors while they listened to these notifications. The results revealed that tempo and pitch have the main effect on the speed and strength (accuracy) of users' cognition and behaviors. The users' characteristics and environments influenced the effects of these musical parameters.

[1]  Li-Wei Ko,et al.  An EEG-based approach for evaluating audio notifications under ambient sounds , 2014, CHI.

[2]  R. Näätänen,et al.  The mismatch negativity (MMN) in basic research of central auditory processing: A review , 2007, Clinical Neurophysiology.

[3]  Ciara M. Greene,et al.  Perceptual Load Induces Inattentional Blindness in Drivers , 2016 .

[4]  Ralf Jung.  Non‐intrusive audio notification in emotion classified background music. , 2010 .

[5]  G. Breukelen.  Analysis of covariance (ANCOVA) , 2010 .

[6]  Céline McKeown.  Auditory Cognition and Human Performance: Research and applications , 2013 .

[7]  A. Friederici,et al.  Children Processing Music: Electric Brain Responses Reveal Musical Competence and Gender Differences , 2003, Journal of Cognitive Neuroscience.

[8]  Jeffrey Heer,et al.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design , 2010, CHI.

[9]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[10]  Neil Salkind,et al.  Encyclopedia of research design , 2010 .

[11]  E. Owens,et al.  An Introduction to the Psychology of Hearing , 1997 .

[12]  Jung-Tai King,et al.  Understanding the Influence of Musical Parameters on Cognitive Responses of Audio Notifications , 2018, CHI Extended Abstracts.

[13]  Uta Boehm,et al.  Attention Theory And Practice , 2016 .

[14]  Heinrich H. Bülthoff,et al.  Use the Right Sound for the Right Job: Verbal Commands and Auditory Icons for a Task-Management System Favor Different Information Processes in the Brain , 2018, CHI.

[15]  J. R. Comstock,et al.  A psychophysiological evaluation of the perceived urgency of auditory warning signals. , 1995, Ergonomics.

[16]  Zhouyu Fu,et al.  A Survey of Audio-Based Music Classification and Annotation , 2011, IEEE Transactions on Multimedia.

[17]  Terrence J. Sejnowski,et al.  Independent Component Analysis of Simulated ERP Data , 2000 .

[18]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[19]  Stavros Garzonis.  Mobile Service Awareness via Auditory Notifications , 2010 .

[20]  Terri L. Bonebright,et al.  Sonific ation Report: Status of the Field and Research Agenda , 2010 .

[21]  Thorsten O. Zander,et al.  Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces , 2010, Brain-Computer Interfaces.

[22]  William W. Gaver.  How Do We Hear in the World?: Explorations in Ecological Acoustics , 1993 .

[23]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing to evaluate information visualization interfaces , 2013, CHI.

[24]  U. Ott,et al.  Using music to induce emotions: Influences of musical preference and absorption , 2008 .

[25]  Nigel Papworth,et al.  Using sound to enhance users' experiences of mobile applications , 2012, AM '12.

[26]  Grégory Leplâtre,et al.  How to Tackle Auditory Interface Aesthetics: Discussion and Case Study , 2004, ICAD.

[27]  Thurmon E. Lockhart,et al.  Aging effect on detectability, criticality and urgency under various auditory conditions , 2015 .

[28]  Christopher Frauenberger,et al.  Auditory display design - An investigation of a design pattern approach , 2009, Int. J. Hum. Comput. Stud..

[29]  E. Schellenberg,et al.  Effects of Musical Tempo and Mode on Arousal, Mood, and Spatial Abilities , 2002 .

[30]  Julieta Ramos-Loyo,et al.  Effects of Musical Tempi on Visual Attention Erps , 2005, The International journal of neuroscience.

[31]  Seiji Yamada,et al.  Artificial subtle expressions: intuitive notification methodology of artifacts , 2010, CHI.

[32]  D. Levitin.  This is Your Brain on Music: Understanding a Human Obsession , 2019 .

[33]  C. Krumhansl.  Rhythm and pitch in music cognition. , 2000, Psychological bulletin.

[34]  R. R. Patterson,et al.  Guidelines for auditory warning systems on civil aircraft , 1982 .

[35]  Katharina Reinecke,et al.  Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness , 2013, CHI.

[36]  Eamonn O'Neill,et al.  Auditory icon and earcon mobile service notifications: intuitiveness, learnability, memorability and preference , 2009, CHI.

[37]  Max L. Wilson,et al.  Using fNIRS in Usability Testing: Understanding the Effect of Web Form Layout on Mental Workload , 2016, CHI.

[38]  Hsinchun Chen,et al.  Visualization of large category map for Internet browsing , 2003, Decis. Support Syst..

[39]  Stephen Brewster,et al.  Experimentally Derived Guidelines for the Creation of Earcons , 2001 .

[40]  Jane F. MacNeil,et al.  Age, music, and driving performance: Detection of external warning sounds in vehicles , 2002 .

[41]  Johan Fagerlönn,et al.  Methods for sound design: a review and implications for research and practice , 2010, Audio Mostly Conference.

[42]  Ranjana K. Mehta,et al.  Neuroergonomics: a review of applications to physical and cognitive work , 2013, Front. Hum. Neurosci..

[43]  John G. Neuhoff,et al.  Sonification Report: Status of the Field and Research Agenda Prepared for the National Science Foundation by members of the International Community for Auditory Display , 1999 .

[44]  I. Winkler,et al.  Involuntary Attention and Distractibility as Evaluated with Event-Related Brain Potentials , 2000, Audiology and Neurotology.

[45]  J Edworthy,et al.  Improving Auditory Warning Design: Relationship between Warning Sound Parameters and Perceived Urgency , 1991, Human factors.