Multi-touch exploration and sonification of line segments

We present SoundLines, a mobile app designed to support children with visual impairments in exercising spatial exploration skills. This is achieved through multi-touch discovery of line segments on touchscreen, coupled with sonification feedback. SoundLines is implemented as a game in which a kitten is guided to find its mother cat by tracing the line connecting them on touchscreen. User study with 4 visually impaired adults assessed the app accessibility and the feasibility of multi-touch exploration, compared to single-touch. While there were limited differences between single-and multi-touch modalities, results indicate that proprioceptive sensing is predominant in multi-touch exploration and that audio feedback provides benefits mostly in single-touch exploration.

[1]  Dragan Ahmetovic,et al.  MathMelodies: Inclusive Design of a Didactic Game to Practice Mathematics , 2014, ICCHP.

[2]  D A Sumikawa.  GUIDELINES FOR THE INTEGRATION OF AUDIO CUES INTO COMPUTER USER INTERFACES , 1985 .

[3]  S. Millar.  Understanding and Representing Space: Theory and Evidence from Studies with Blind and Sighted Children , 1994 .

[4]  João Guerreiro,et al.  AudioFunctions.web: Multimodal Exploration of Mathematical Function Graphs , 2019, W4A.

[5]  Lisa Anthony,et al.  Using gamification to motivate children to complete empirical studies in lab environments , 2013, IDC.

[6]  Gregory Kramer,et al.  Auditory Display: Sonification, Audification, And Auditory Interfaces , 1994 .

[7]  Kyle Montague,et al.  TabLETS Get Physical: Non-Visual Text Entry on Tablet Devices , 2015, CHI.

[8]  Steven M. Casey,et al.  Cognitive Mapping by the Blind , 1978 .

[9]  Mahdi Tavakoli,et al.  Preliminary testing by adults of a haptics-assisted robot platform designed for children with physical impairments to access play , 2018, Assistive technology : the official journal of RESNA.

[10]  Hideki Koike,et al.  EdgeSonic: image feature sonification for the visually impaired , 2011, AH '11.

[11]  Nahyun Kwon,et al.  Touch Screen Exploration of Visual Artwork for Blind People , 2021, WWW.

[12]  Amy Hurst,et al.  VizTouch: automatically generated tactile visualizations of coordinate spaces , 2012, TEI.

[13]  John A. Gardner,et al.  Scientific Diagrams Made Easy with IVEOTM , 2006, ICCHP.

[14]  A. Kappers,et al.  The Influence of Picture Size on Recognition and Exploratory Behaviour in Raised-Line Drawings , 2008, Perception.

[15]  T. Dick,et al.  Issues and Aids for Teaching Mathematics To the Blind , 1997 .

[16]  Cheng-Lung Lee,et al.  Assessment of a simple obstacle detection device for the visually impaired. , 2014, Applied ergonomics.

[17]  Dragan Ahmetovic,et al.  Remote Usage Data Collection and Analysis for Mobile Accessibility Applications , 2021, 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops).

[18]  M. Blades,et al.  Understanding spatial concepts at the geographic scale without the use of vision , 1997 .

[19]  Linda Pring,et al.  Psychological characteristics of children with visual impairments: learning, memory and imagery , 2008 .

[20]  Derrick W. Smith,et al.  The Role and Characteristics of Tactile Graphics in Secondary Mathematics and Science Textbooks in Braille , 2012 .

[21]  M. Sheldon,et al.  The use and interpretation of the Friedman test in the analysis of ordinal-scale data in repeated measures designs. , 1996, Physiotherapy research international : the journal for researchers and clinicians in physical therapy.