Speaking through pictures: images vs. icons

People with aphasia, a condition that impairs the ability to understand or generate written or spoken language, are aided by assistive technology that helps them communicate through a vocabulary of icons. These systems are akin to language translation systems, translating icon arrangements into spoken or written language and vice versa. However, these icon-based systems have little vocabulary breadth or depth, making it difficult for people with aphasia to apply their usage to multiple real world situations. Pictures from the web are numerous, varied, and easily accessible and thus, could potentially address the small size issues of icon-based systems. We present results from two studies that investigate this potential and demonstrate that images can be as effective as icons when used as a replacement for English language communication. The first study uses elderly subjects to investigate the efficacy of images vs. icons in conveying word meaning; the second study examines the retention of word-level meaning by both images and icons with a population of aphasics. We conclude that images collected from the web are as functional as icons in conveying information and thus, are feasible to use in assistive technology that supports people with aphasia.

[1]  Jiska Wiegers,et al.  A computerised communication aid for people with aphasia , 2005, Disability and rehabilitation.

[2]  Henrik Danielsson,et al.  Pictures as language , 2001 .

[3]  Joanna McGrenere,et al.  The Field Evaluation of a Mobile Digital Image Communication Application Designed for People with Aphasia , 2008, TACC.

[4]  J. Hochberg,et al.  Pictorial recognition as an unlearned ability: a study of one child's performance. , 1962, The American journal of psychology.

[5]  Lars Kjelldahl,et al.  A Survey of Some Perceptual Features for Computer Graphics and Visualization , 2003 .

[6]  S. Nadel,et al.  A FIELD EXPERIMENT IN RACIAL1 PSYCHOLOGY2 , 1937 .

[7]  G. Miller,et al.  A Semantic Network of English Verbs , 1998 .

[8]  Laura Thorburn,et al.  Ability of Subjects With Aphasia to Visually Analyze Written Language, Pantomime, and Iconographic Symbols , 1995 .

[9]  Christina M. Lucia.  Toward Developing a Model of Music Therapy Intervention in the Rehabilitation of Head Trauma Patients , 1987 .

[10]  Marilyn Tremaine,et al.  Participatory design with proxies: developing a desktop-PDA system to support people with aphasia , 2006, CHI.

[11]  Robert T. Wertz,et al.  Computer-based visual communication in aphasia , 1989, Neuropsychologia.

[12]  M. Schlossberg.  The Assessment of Aphasia and Related Disorders. 2nd ed. , 1984 .

[13]  D. C. Beardslee,et al.  Readings in perception , 1958 .

[14]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[15]  T. A. Ryan,et al.  Speed of perception as a function of mode of representation. , 1956, The American journal of psychology.

[16]  I. T. Draper.  THE ASSESSMENT OF APHASIA AND RELATED DISORDERS , 1973 .

[17]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[18]  Stephen Wallace,et al.  Figure and Ground , 1982 .

[19]  G. Mcreddie.  Aphasia , 1868, The Indian medical gazette.

[20]  Maria M. Klawe,et al.  The participatory design of a sound and image enhanced daily planner for people with aphasia , 2004, CHI.