Design and evaluation of nonverbal sound-based input for those with motor handicapped

Most personal computing interfaces rely on the users’ ability to use their hand and arm movements to interact with on-screen graphical widgets via mainstream devices, including keyboards and mice. Without proper assistive devices, this style of input poses difficulties for motor-handicapped users. We propose a sound-based input scheme enabling users to operate Windows’ Graphical User Interface by producing hums and fricatives through regular microphones. Hierarchically arranged menus are utilized so that only minimal numbers of different actions are required at a time. The proposed scheme was found to be accurate and capable of responding promptly compared to other sound-based schemes. Being able to select from multiple item-selecting modes helps reducing the average time duration needed for completing tasks in the test scenarios almost by half the time needed when the tasks were performed solely through cursor movements. Still, improvements on facilitating users to select the most appropriate modes for desired tasks should improve the overall usability of the proposed scheme. Implications for Rehabilitation Using typical means for computer input like keyboard and mouse may pose challenges to certain motor-handicapped users who are unable to control their hands or arms. Non-verbal, humming sound input is a good candidate solution that utilizes standard functions, hence economical and language independent. This Desktop Access with Non-verbal Sound Input is an assistive device that allows motor-impaired users to command and navigate the computer with minimal training.