The Use of Multimodal Representation in Icon Interpretation

Identifying icon functions differs from naming pictures in that strong semantic links between pictures and their names have been formed over a long period of time whereas the meaning of icons has often to be learned. This paper examines roles of icon characteristics such as complexity, concreteness, familiarity and aesthetic appeal in determining how easily icons can be learned and identified. The role of these characteristics is seen as dynamic, changing as the user learns the icon set. It is argued that the way in which users learn icon meanings is similar to the processes involved in language learning. Icon meanings are learned by drawing on rich multimodal representations which are the result of our world experience. This approach could lead to a better understanding of how multimodal information can be most usefully presented on interfaces.