Haptic Wave: A Cross-Modal Interface for Visually Impaired Audio Producers

We present the Haptic Wave, a device that allows cross-modal mapping of digital audio to the haptic domain, intended for use by audio producers/engineers with visual impairments. We describe a series of participatory design activities adapted to non-sighted users where the act of prototyping facilitates dialog. A series of workshops scoping user needs, and testing a technology mock up and lo-fidelity prototype fed into the design of a final high-spec prototype. The Haptic Wave was tested in the laboratory, then deployed in real world settings in recording studios and audio production facilities. The cross-modal mapping is kinesthetic and allows the direct manipulation of sound without the translation of an existing visual interface. The research gleans insight into working with users with visual impairments, and transforms perspective to think of them as experts in non-visual interfaces for all users.

[1]  P. Bach-y-Rita,et al.  Sensory substitution and the human–machine interface , 2003, Trends in Cognitive Sciences.

[2]  Jakob Bak,et al.  Motors, Music and Motion , 2015, Tangible and Embedded Interaction.

[3]  Youn-Kyung Lim,et al.  Handscope: enabling blind people to experience statistical graphics on websites through haptics , 2011, CHI.

[4]  Atau Tanaka,et al.  Form Follows Sound: Designing Interactions from Sonic Memories , 2015, CHI.

[5]  Julio Abascal,et al.  Universal accessibility as a multimodal design issue , 2007, CACM.

[6]  Stephen A. Brewster,et al.  Multimodal 'eyes-free' interaction techniques for wearable devices , 2003, CHI '03.

[7]  Sharon L. Oviatt,et al.  Ten myths of multimodal interaction , 1999, Commun. ACM.

[8]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[9]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[10]  Ali Israr,et al.  TeslaTouch: electrovibration for touch surfaces , 2010, UIST.

[11]  Patrick Baudisch,et al.  Blindsight: eyes-free access to mobile phones , 2008, CHI.

[12]  Enrico Pontelli,et al.  Exploring tangible collaborative distance learning environments for the blind and visually impaired , 2013, CHI Extended Abstracts.

[13]  Charles Camp Thinking through Material Culture: An Interdisciplinary Perspective , 2008 .

[14]  Sigrid Norris,et al.  Analyzing Multimodal Interaction: A Methodological Framework , 2004 .

[15]  Karon E. MacLean,et al.  Manipulating music: multimodal interaction for DJs , 2004, CHI '04.

[16]  Enrico Pontelli,et al.  Multimodal Presentation of Two-Dimensional Charts: An Investigation Using Open Office XML and Microsoft Excel , 2010, TACC.

[17]  James D. Hollan,et al.  Direct Manipulation Interfaces , 1985, Hum. Comput. Interact..

[18]  Allison Druin,et al.  Technology probes: inspiring design for and with families , 2003, CHI '03.

[19]  Sriram Subramanian,et al.  Talking about tactile experiences , 2013, CHI.

[20]  Andrew Sears,et al.  Representing users in accessibility research , 2012, TACC.

[21]  Atau Tanaka,et al.  Co-production and Co-creation: Creative Practice in Social Inclusion , 2010, ECS.

[22]  Ephraim P. Glinert,et al.  Improving GUI accessibility for people with low vision , 1995, CHI '95.

[23]  Max V. Mathews,et al.  The PLANK: Designing a simple Haptic Controller , 2002, NIME.

[24]  Oussama Metatla,et al.  Constructing relational diagrams in audio: the multiple perspective hierarchical approach , 2008, Assets '08.

[25]  Atau Tanaka,et al.  HapticWave: Presenting the Multiple Voices, Artefacts and Materials of a Design Research Project , 2015 .

[26]  Olivier Bau,et al.  The A20: Interactive Instrument Techniques for Sonic Design Exploration , 2013 .

[27]  Chung Hyuk Park,et al.  Real world haptic exploration for telepresence of the visually impaired , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[28]  D. Lewkowicz,et al.  A dynamic systems approach to the development of cognition and action. , 2007, Journal of cognitive neuroscience.

[29]  William W. Gaver Making spaces: how design workbooks work , 2011, CHI.

[30]  Calle Sjöström,et al.  Non-Visual Haptic Interaction Design - Guidelines and Applications , 2002 .

[31]  Richard E. Ladner,et al.  Usable gestures for blind people: understanding preference and performance , 2011, CHI.

[32]  Stephen A. Brewster,et al.  Setting the Standards for Haptic and Tactile Interactions: ISO's Work , 2010, EuroHaptics.

[33]  Bill Verplank,et al.  Can Haptics Make New Music ? - Fader and Plank Demos , 2011, NIME.

[34]  Nadia Bianchi-Berthouze,et al.  Tactile perceptions of digital textiles: a design research approach , 2013, CHI.

[35]  Ian Oakley,et al.  Putting the feel in ’look and feel‘ , 2000, CHI.