Multimodal messages: the pen and voice opportunity

Abstract Analyses of the costs and benefits of asynchronous communication, and the complementary properties of writing and speech, are used to predict that messages containing both writing and speech will be more communicative than either medium alone. Two experimental studies of asynchronous messaging are presented. Both experiments examine the use of pen-and-voice messages, that is voice messages attached to ‘scribbled’, i.e., uninterpreted text. The control conditions were voice messages alone, equivalent to an answerphone, and scribbled messages alone, equivalent to a fax. In Experiment 1 the visual component of the pen-and-voice messages was static, in Experiment 2 users could record short ‘movies’ including speech and pen movements over a document surface. Users showed a significant preference for the pen-and-voice messages in both experiments. In Experiment 2 half the number of pen-and-voice messages were required to achieve the same task performance as in the control conditions. It is concluded that dynamic pen-and-voice messages have considerable potential advantages over current single medium asynchronous communication facilities such as fax, answerphone, voicemail and e-mail.