AudioFunctions.web: Multimodal Exploration of Mathematical Function Graphs

We present AudioFunctions.web, a web app that uses sonification, earcons and speech synthesis to enable blind people to explore mathematical function graphs. The system is designed for personalized access through different interfaces (touchscreen, keyboard, touchpad and mouse) on both mobile and traditional devices, in order to better adapt to different user abilities and preferences. It is also publicly available as a web service and can be directly accessed from the teaching material through a hypertext link. An experimental evaluation with 13 visually impaired participants highlights that, while the usability of all the presented interaction modalities is high, users with different abilities prefer different interfaces to interact with the system. It is also shown that users with higher level of mathematical education are capable of better adapting to interaction modalities considered more difficult by others.

[1]  Youn-Kyung Lim,et al.  Handscope: enabling blind people to experience statistical graphics on websites through haptics , 2011, CHI.

[2]  Dragan Ahmetovic,et al.  Math Melodies: Supporting Visually Impaired Primary School Students in Learning Math , 2017, W4A.

[3]  Nancy Alajarmeh.  Doing math: mathematics accessibility issues , 2012, W4A.

[4]  Franco Lepore,et al.  Early- and Late-Onset Blind Individuals Show Supra-Normal Auditory Abilities in Far-Space , 2004, Current Biology.

[5]  Dragan Ahmetovic,et al.  MathMelodies: Inclusive Design of a Didactic Game to Practice Mathematics , 2014, ICCHP.

[6]  Piero Mussio,et al.  A multimodal interactive system to create and explore graph structures , 2008, CHI Extended Abstracts.

[7]  M. H. van Emden,et al.  Interval arithmetic: From principles to implementation , 2001, JACM.

[8]  Taliesin L. Smith,et al.  Design and Evaluation of a Multimodal Science Simulation , 2018, ASSETS.

[9]  Dragan Ahmetovic,et al.  AudioFunctions: Eyes-Free Exploration of Mathematical Functions on Tablets , 2014, ICCHP.

[10]  Volker Sorge,et al.  Towards universal rendering in MathJax , 2016, W4A.

[11]  Volker Sorge,et al.  Towards making mathematics a first class citizen in general screen readers , 2014, W4A.

[12]  Cristian Bernareggi,et al.  On the Evaluation of Novel Sonification Techniques for Non-Visual Shape Exploration , 2017, ACM Trans. Access. Comput..

[13]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[14]  Dragan Ahmetovic,et al.  MathMelodies 2: a Mobile Assistive Application for People with Visual Impairments Developed with React Native , 2018, ASSETS.

[15]  Stephen A. Brewster,et al.  Constructing sonified haptic line graphs for the blind student: first steps , 2000, Assets '00.

[16]  Catherine S. Fichten,et al.  Accessibility of e-Learning and Computer and Information Technologies for Students with Visual Impairments in Postsecondary Education , 2009 .

[17]  Chandrika Jayant,et al.  A Survey of Math Accessibility For Blind Persons and An Investigation on Text / Math Separation , 2006 .

[18]  Dragan Ahmetovic,et al.  Axessibility: a LaTeX Package for Mathematical Formulae Accessibility in PDF Documents , 2018, ASSETS.

[19]  Kathleen F. McCoy,et al.  Evaluating the accessibility of line graphs through textual summaries for visually impaired users , 2014, ASSETS.

[20]  Kim Marriott,et al.  GraCALC: An Accessible Graphing Calculator , 2015, ASSETS.

[21]  John A. Gardner,et al.  Scientific Diagrams Made Easy with IVEOTM , 2006, ICCHP.

[22]  João Guerreiro,et al.  Virtual Navigation for Blind People: Building Sequential Representations of the Real-World , 2017, ASSETS.

[23]  Cristian Bernareggi,et al.  Eyes-free Exploration of Shapes with Invisible Puzzle , 2015, ASSETS.

[24]  Mark D. Gross,et al.  FluxMarker: Enhancing Tactile Graphics with Dynamic Tactile Markers , 2017, ASSETS.

[25]  Meera M. Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles (Abstract only) , 1989, SGCH.

[26]  Richard E. Ladner,et al.  Automating tactile graphics translation , 2005, Assets '05.

[27]  Timo Götzelmann,et al.  Visually Augmented Audio-Tactile Graphics for Visually Impaired People , 2018, ACM Trans. Access. Comput..

[28]  John A. Gardner,et al.  Access by Blind Students and Professionals to Mainstream Math and Science , 2002, ICCHP.

[29]  Nadir Murru,et al.  An Automatized Method Based on LaTeX for the Realization of Accessible PDF Documents Containing Formulae , 2018, ICCHP.

[30]  Kathleen F. McCoy,et al.  Multimodal Deep Learning using Images and Text for Information Graphic Classification , 2018, ASSETS.

[31]  Marcos Serrano,et al.  Identifying how Visually Impaired People Explore Raised-line Diagrams to Improve the Design of Touch Interfaces , 2017, CHI.

[32]  Bruce N. Walker,et al.  SONIFICATION SANDBOX: A GRAPHICAL TOOLKIT FOR AUDITORY GRAPHS , 2003 .

[33]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[34]  Kening Zhu,et al.  ColorTact: A Finger Wearable Audio-tactile Device Using Customizable Color Tagging , 2018, UbiComp/ISWC Adjunct.

[35]  Kenneth E. Barner,et al.  Automatic production of tactile graphics from scalable vector graphics , 2005, Assets '05.

[36]  Torsten Wörtwein,et al.  Interactive Web-based Image Sonification for the Blind , 2015, ICMI.