Chapter 9 – Images, words, and gestures

This chapter addresses the relationship between visual information and verbal or textual information: images, words, and gestures. Most of the visualizations are not purely graphical; they are composites, combining images with text or spoken language. A particularly thorny but interesting problem is whether or not one should be using visual languages to program computers. It has been suggested that visual programming languages may make it easier for “nonprogrammers” to program computers. The chapter discusses some of the advantages of visual representations, such as better comprehension of patterns and spatial relationships in general. The material represents a number of conclusions about how one should design easy-to-learn computer programming languages. They should be hybrids of visual and natural language codes. Structure should be presented visually, and perhaps also created visually using direct manipulation techniques.