Multimodal interfaces: new solutions to the problem of computer accessibilty for the blind

This paper examines how multimodal interfaces can improve the accessibility of software application for blind users. The approach, which consists of translating visual interaction forms through non visual modalities, can’t be successfully applied to graphical interfaces. Optimizing interface for the blind involves rethinking paradigms and building tbe application interface on another base. Multimodal interfaces open new avenues for research and development into this area. This paper discusses these promising perspectives through a concrete example: a prototype multimodal text editor that has been developed in a research project between INSERM and CNRS. INTRODUCTION Computers open up a new opportunity for the social and professional integration of people with a visual handicap. People who are blind can carry on with their daily work using computers tools that have been adapted to their special requirements. They can take notes, receive and send faxes or E-mails, edit texts and consult dictionaries. Nevertheless, this great potential seemsto be far from fully realised, mainly because of the human factors in the interfaces [1]. Thus a challenge for rehabilitation technology is to make access products for the blind as user-friendly as are the graphical interface-based applications for sighted people. This implies basing their design on specific features of non-visual communication, rather than simply trying to adapt visual dialogue methods. Multimodal interfaces open new avenues for research into the solution of these problems. This paper discusses and illustrates these promising perspectives. APPROACHES TO ADAPTING VISUAL INTERFACES Software production has shifted from character-based interfaces to graphical user interfaces (GUI) over the past few years. This change is even more radical for the blind than for sighted users because it makes the access techniques previously used totally Permlssmn to copy without fee all or part of thm material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice end the title of the pubkcation and its date appear, and notice IS gwen that copying is by perm!swon of the Association for Computtng Machinery. To copy otherwme, or to republish, requires a fee andlor specific permission. CH194 Companion-4/94 Boston, Massachusetts USA o 1994 ACM 0-89791 -651 -4/94 /0267 . ..$3.50 Dominique BURGER INSERM UPMC B23, 75252 PARIS Cedex 05 FRANCE Tel :33.1.44.27.34.35 inadequate. Indeed they are based on building a program able to read the characters contained in the screen memory. The first consequence is that developers have to find means of accessing data to be displayed on the screen. Fortunately, computer scientist have to rely on well organised Operating Systems to develop sophisticated interfaces like GUI’s, and these are composed of software-independent components that communicate with each other by exchanging messages. This allowed the building of the so called Off-Screen Model (OSM) first develop by Berkeley Systems. The user does not directly access the screen, but a model of it that has been rebuild within the computer memory [2]. The access program can then retain representation of text, icons, controls and other information in a form that is close to the semantic level. It can rdso capture information that is not visible on the screen, which can be usefid to the user[3]. Fig. 1 illustrates two approaches to the problem of adapting software application for the blind. The traditional approach, with character-based interfaces was to translate the visual messages on the screen into a non-visual form. Another approach that take advantage of reeent technology build alternative interfaces for people with special needs. Visual Intwactlon Farm Tmnsfdkm ~7 Non Viaml Intoractkm Fwrn Transletlon Approach I& & ( Non Vbual Intwfaca )1 I Alternative Approach I Fig. 1: Approached for Adapting Vieual Interface NOVAE: A PROJECT TO EXPLORE MULTIMODAL SOLUTIONS The ultimate advancement in research and development is still to produce interface systems that make it possible for the user to alternate or combine several channels to interact with the computer. Such interfaces are refereed as multimodal Multimodal interfaces have features that could help to overcome some of the limitations of existing solutions for the blind. In lparticular, they could provide: . better adaptation to user’s needs; . intuitive, concise and quick interaction modes; . easier learning and reduction of memorisation efforts; . increase power expression.