Audio Feedback Design Principles for Hand Gestures in Audio-Only Games

The design of audio feedback for touch-less gesture interaction with audio-only environments requires a structured approach. We present design principles for responsive audio feedback geared towards hand gestures in audio-only games, focusing on diegetic environmental feedback before, during, and after gestures. Illustrations for the principles are drawn from a project investigating the usefulness of different designs for encouraging user participation and maintaining immersion in audio-only games. The findings indicate that replacing explicit audio instructions for hand positions and movements with responsive audio feedback for suggesting interaction methods using environmental story-related audio cues leads to measurably higher immersion. Principles for designing immersive audio feedback responsive to hand gesture interaction in a 3-dimensional space also have implications for virtual reality and the blind community's access to motion-controlled games.

[1]  J. Murray.  Hamlet on the Holodeck , 1997 .

[2]  Antony Radford,et al.  Games and Learning about Form in Architecture , 2000, Proceedings of the 15th International Conference on Education and Research in Computer Aided Architectural Design in Europe (eCAADe).

[3]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[4]  Kenton O'Hara,et al.  On the naturalness of touchless: Putting the “interaction” back into NUI , 2013, TCHI.

[5]  Donald A. Norman,et al.  Natural user interfaces are not natural , 2010, INTR.

[6]  J. Murray.  Hamlet on the Holodeck , 1997 .

[7]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[8]  Nigel Papworth,et al.  Beowulf: an audio mostly game , 2007, ACE '07.

[9]  Jeanne H. Brockmyer,et al.  The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game Playing: Response to Reviews , 2009, Interacting with computers.

[10]  Ian Parberry,et al.  Toward mobile entertainment: A paradigm for narrative-based audio only games , 2007, Sci. Comput. Program..

[11]  Rex Hartson,et al.  The UX book, process and guidelines for ensuring a quality user experience by Rex Hartson and Pardha S. Pyla , 2012, SOEN.

[12]  Terry Winograd,et al.  Understanding computers and cognition , 1986 .

[13]  Nigel Papworth.  iSpooks: an audio focused game design , 2010, Audio Mostly Conference.

[14]  Jennifer Ann Rode,et al.  Using a large projection screen as an alternative to head-mounted displays for virtual environments , 2000, CHI.