It's All About the Message: Visual Experience is a Precursor to Accurate Auditory Interaction

Anecdotal evidence suggests there is a disjoint between the interaction experiences of sighted and visually disabled web users. However, we propose the converse and suggest that this disjoint is created by the lack of understanding of the interplay between the two domains. Current research shows that there is one single locus of attention at a given time in the context of web interaction, and therefore sighted users form a serialisation of the things they look at and pay attention - an exemplar of which can be seen in eye movement sequences of users. We also suggest that web designers have a narrative in mind to be experienced by users, and they create a visual sequence they wish their audience to perceive for supporting this narrative. However, this sequence is typically lost when we move from visual presentations to auditory ones. Current audio interactions centre around page linearisation based on the sequence of the underlying source code. This linearisation typically falls short of the kind of comprehensive interaction which can be expected in the visual domain. In this paper, we use an eye tracking dataset to illustrate that the linearisation of web page component based on the underlying source code differs from what is experienced by sighted users. We then show that the web experience of visually disabled users can be improved by re-ordering the most commonly used web page components based on the order in which they are used. We also suggest that it is critical to conduct formative experimentation with sighted users to establish a visual narrative and serialisation, thereby informing the design of the auditory conversation.