Optimal audio-visual representations for illiterate users of computers

We present research leading toward an understanding of the optimal audio-visual representation for illustrating concepts for illiterate and semi-literate users of computers. In our user study, which to our knowledge is the first of its kind, we presented to 200 illiterate subjects each of 13 different health symptoms in one representation randomly selected among the following ten: text, static drawings, staticphotographs, hand-drawn animations, and video, each with and without voice annotation. The goal was to see how comprehensible these representation types were for an illiterate audience. We used a methodology for generating each of the representations tested in a way that fairly stacks one representational type against the others. Our main results are that (1) voice annotation generally helps in speed of comprehension, but bimodal audio-visual information can be confusing for the target population; (2) richer information is not necessarily better understood overall; (3) the relative value of dynamic imagery versus static imagery depends on various factors. Analysis of these statistically significant results and additional detailed results are also provided.

[1]  Media Lab Asia.  Designing for the Indian rural population : Interaction design challenges , 2002 .

[2]  Sally Grisedale,et al.  Designing a graphical user interface for healthcare workers in rural India , 1997, CHI.

[3]  Henry Dreyfuss.  Symbol Sourcebook: An Authoritative Guide to International Graphic Symbols , 1972 .

[4]  Albert T. Corbett,et al.  Assessing dynamics in computer-based instruction , 1996, CHI.

[5]  John T. Stasko,et al.  Evaluating animations as student aids in learning computer algorithms , 1999, Comput. Educ..

[6]  Brian K. Smith,et al.  Visualizing health: imagery in diabetes education , 2003, DUX '03.

[7]  Susan Palmiter,et al.  Animated Demonstrations for Learning Procedural Computer-Based Tasks , 1993, Hum. Comput. Interact..

[8]  Johanna D. Moore,et al.  Proceedings of the Conference on Human Factors in Computing Systems , 1989 .

[9]  Jim Bolek,et al.  Development of a symbol system for use in the health care industry , 2005, DUX '05.

[10]  William K. Horton,et al.  The Icon Book: Visual Symbols for Computer Systems and Documentation , 1994 .

[11]  Barbara Tversky,et al.  The (in)effectiveness of animation in instruction , 2001, CHI Extended Abstracts.

[12]  Gregory D. Abowd,et al.  Storyboarding: an empirical determination of best practices and effective guidelines , 2006, DIS '06.

[13]  Sugata Mitra,et al.  Self organising systems for mass computer literacy: Findings from the ‘hole in the wall’ experiments , 2005 .

[14]  Lloyd P. Rieber,et al.  The effects of computer animation on adult learning and retrieval tasks , 1990 .

[15]  Aaron Marcus,et al.  Cross-GUI Handbook: For Multiplatform User Interface Design , 1994 .

[16]  William W. Gaver.  Auditory Icons: Using Sound in Computer Interfaces , 1986, Hum. Comput. Interact..

[17]  Wolfgang Schnotz,et al.  Individual and co-operative learning with interactive animated pictures , 1999 .

[18]  Lloyd P. Rieber,et al.  The Effects of Computer Animated Elaboration Strategies and Practice on Factual and Application Learning in an Elementary Science Lesson , 1989 .

[19]  Alain Breuleux,et al.  Effect of animation in enhancing descriptive and procedural texts in a multimedia learning environment , 1996 .

[20]  John W. Finnegan,et al.  007 - Acting and drawing for animation , 2004, SIGGRAPH '04.

[21]  Alain Breuleux,et al.  Effect of Animation in Enhancing Descriptive and Procedural Texts in a Multimedia Learning Environment , 1996, J. Am. Soc. Inf. Sci..

[22]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[23]  David Salesin,et al.  Schematic storyboarding for video visualization and editing , 2006, SIGGRAPH '06.

[24]  Thomas Strothotte,et al.  An Approach to Help Functionally Illiterate People with Graphical Reading Aids , 2001 .

[25]  William Horton,et al.  Designing icons and visual symbols , 1996, CHI Extended Abstracts.

[26]  S V Thompson,et al.  The effect of animated diagrams on the understanding of a mathematical demonstration in 11- to 14-year-old pupils. , 1990, The British journal of educational psychology.

[27]  Aaron J. Marcus,et al.  Icon and symbol design issues for graphical user interfaces , 1996 .

[28]  Jakob Nielsen,et al.  International users interface , 1996 .

[29]  P. Krishna Reddy,et al.  eSagu 1 : An IT based personalized agricultural extension system prototype - analysis of 51 Farmers' case studies , 2006 .

[30]  Protima Banerjee,et al.  Book Review: About Face 2.0: The essentials of interaction design , 2004, Inf. Vis..

[31]  David E. Kieras,et al.  Diagrammatic Displays for Engineered Systems: Effects on Human Performance in Interacting with Malfunctioning Systems , 1992, Int. J. Man Mach. Stud..

[32]  Susan Palmiter,et al.  Animated Demonstrations vs Written Instructions for Learning Procedural Tasks: A Preliminary Investigation , 1991, Int. J. Man Mach. Stud..

[33]  David Werner,et al.  Where There Is No Doctor: A Village Health Care Handbook, Revised Edition , 1987 .

[34]  Aaron Marcus,et al.  Icons, symbols, and signs: visible languages to facilitate communication , 2003, INTR.

[35]  Tapan S. Parikh,et al.  Design studies for a financial management system for micro-credit groups in rural india , 2002, CUU '03.

[36]  Apala Lahiri Chavan,et al.  Design considerations for a financial management system for rural, semi-literate users , 2003, CHI Extended Abstracts.

[37]  Kentaro Toyama,et al.  Text-Free User Interfaces for Illiterate and Semi-Literate Users , 2006, 2006 International Conference on Information and Communication Technologies and Development.

[38]  Barbara Tversky,et al.  Animation: can it facilitate? , 2002, Int. J. Hum. Comput. Stud..

[39]  Ok-choon Park,et al.  Selective use of animation and feedback in computer-based instruction , 1992 .

[40]  Brad Mehlenbacher.  Proceedings of the 14th annual international conference on Systems documentation: Marshaling new technological forces: building a corporate, academic, and user-oriented triangle , 1996 .

[41]  Stephen J. Payne,et al.  Animated Demonstrations for Exploratory Learners , 1992, Interact. Comput..

[42]  Mitchell J. Nathan,et al.  A theory of algebra-word-problem comprehension and its implications for the design of learning environments. , 1992 .

[43]  Lloyd P. Rieber,et al.  Effects of textual and animated orienting activities and practice on learning from computer-based instruction , 1988 .

[44]  William W. Gaver.  The SonicFinder: An Interface That Uses Auditory Icons , 1989, Hum. Comput. Interact..

[45]  Bharathi Pitti,et al.  Text-Free UI for Employment Search , 2005 .

[46]  Alan Cooper,et al.  About Face 3: the essentials of interaction design , 1995 .

[47]  Theo van Leeuwen,et al.  Reading Images: The Grammar of Visual Design , 1996 .

[48]  Danny Dowhal.  A Seven-Dimensional Approach to Graphics , 1996, SIGDOC.