Knowing by ear: leveraging human attention abilities in interaction design

In a world in which intelligent technologies are integrated in everyday objects and environments, users are at risk of being overburdened with information and interaction possibilities. Calm technology therefore aims at designing interactions that may reside in the periphery of the user’s attention and only shift to the center of the attention when required. However, for such designs to be effective, a detailed understanding of human attention abilities is needed. In this paper, we therefore present a qualitative study on the everyday periphery of the attention. As we expected, we found that sound plays a major role in this, which supports our approach to use interactive sonification as an interaction style for peripheral interaction. We present a range of rich examples of everyday situations that lay out the design space for peripheral interaction and support these findings by describing three initial designs that use interactive sonification for peripheral interaction.

[1]  Mark Weiser,et al.  The computer for the 21st Century , 1991, IEEE Pervasive Computing.

[2]  M. Weiser,et al.  THE COMING AGE OF CALM TECHNOLOGY[1] , 1996 .

[3]  Dennis H. Rouvray,et al.  Beyond calculation: The next fifty years of computing , 1997 .

[4]  W. James,et al.  The Principles of Psychology. , 1983 .

[5]  Panos Markopoulos.  A Design Framework for Awareness Systems , 2009, Awareness Systems.

[6]  H. Pashler.  The Psychology of Attention , 1997 .

[7]  Thomas Hermann,et al.  Supplementary Material for "An Introduction to Interactive Sonification (Guest Editors' Introduction)" , 2005 .

[8]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[9]  A. Treisman.  VERBAL CUES, LANGUAGE, AND MEANING IN SELECTIVE ATTENTION. , 1964, The American journal of psychology.

[10]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[11]  E. C. Cmm,et al.  on the Recognition of Speech, with , 2008 .

[12]  Clare-Marie Karat,et al.  CHI 98 Conference Summary on Human Factors in Computing Systems , 1998, CHI Conference Summary.

[13]  Pieter Jan Stappers,et al.  Contextmapping: experiences from practice , 2005 .

[14]  William W. Gaver.  What in the World Do We Hear? An Ecological Approach to Auditory Event Perception , 1993 .

[15]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[16]  Sriram Subramanian,et al.  Talking about tactile experiences , 2013, CHI.

[17]  Anton Nijholt,et al.  Design for the periphery , 2010 .

[18]  Thomas Hermann,et al.  An introduction to interactive sonification , 2005 .

[19]  Bruce N. Walker,et al.  Creating Functional and Livable Soundscapes for Peripheral Monitoring of Dynamic Data , 2004, ICAD.

[20]  N. Moray.  Attention in Dichotic Listening: Affective Cues and the Influence of Instructions , 1959 .

[21]  Elizabeth D. Mynatt,et al.  Designing audio aura , 1998, CHI.

[22]  Wendy E. Mackay,et al.  Awareness Systems - Advances in Theory, Methodology and Design , 2009, Human-Computer Interaction Series.

[23]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[24]  D. Kahneman,et al.  Attention and Effort , 1973 .

[25]  Hiroshi Ishii,et al.  ambientROOM: integrating ambient media with architectural space , 1998, CHI Conference Summary.

[26]  P. Liamputtong.  Qualitative Research Methods , 2005 .

[27]  William W. Gaver,et al.  Design: Cultural probes , 1999, INTR.

[28]  Andy Hunt,et al.  Guest Editors' Introduction: An Introduction to Interactive Sonification , 2005, IEEE Multim..

[29]  Tara Matthews,et al.  A toolkit for managing user attention in peripheral displays , 2004, UIST '04.

[30]  Hsiu-Fang Hsieh,et al.  Three Approaches to Qualitative Content Analysis , 2005, Qualitative health research.

[31]  Berry Eggen,et al.  Making Sense of What Is Going on 'Around': Designing Environmental Awareness Information Displays , 2009, Awareness Systems.

[32]  J G Hollands,et al.  ENGINEERING PSYCHOLOGY AND HUMAN PERFORMANCE - THIRD EDITION , 2000 .

[33]  Peter J. Denning,et al.  Beyond calculation - the next fifty years of computing , 1997 .