Mental imagery in program design and visual programming

There is widespread anecdotal evidence that expert programmers make use of visual mental images when they are designing programs. This evidence is used to justify the use of diagrams and visual programming languages during software design. This paper reports the results of two studies. In the first, expert programmers were directly questioned regarding the nature of their mental representations while they were engaged in a design task. This investigative technique was used with the explicit intention of eliciting introspective reports of mental imagery. In the second, users of a visual programming language responded to a questionnaire in which they were asked about cognitive processes. The resulting transcripts displayed a considerable number of common elements. These suggests that software design shares many characteristics of more concrete design disciplines. The reports from participants in the two studies, together with previous research into imagery use, indicate potential techniques for further investigation of software development support tools and design strategies.

[1]  Thomas H. Leahey,et al.  Emerging Visions of the Aesthetic Process: Psychology, Semiology, and Philosophy. , 1994 .

[2]  R. K. Lindsay.  Images and inference , 1988, Cognition.

[3]  Graham J. Hitch,et al.  Two types of representation in visual memory: Evidence from the effects of stimulus contrast on image combination , 1995, Memory & cognition.

[4]  Marian Petre,et al.  Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework , 1996, J. Vis. Lang. Comput..

[5]  Mark A. McDaniel,et al.  Imagery and Cognition , 1991 .

[6]  David J. Gilmore,et al.  Comprehension and Recall of Miniature Programs , 1984, Int. J. Man Mach. Stud..

[7]  Alan F. Blackwell,et al.  Visual Programming in the Wild: A Survey of LabVIEW Programmers , 2001, J. Vis. Lang. Comput..

[8]  George A. Miller,et al.  Metaphor and Thought: Images and models, similes and metaphors , 1993 .

[9]  Masaki Suwa,et al.  What do architects and students perceive in their design sketches? A protocol analysis , 1997 .

[10]  S. Kosslyn.  Image and Brain: The Resolution of the Imagery Debate , 1994, Journal of Cognitive Neuroscience.

[11]  G. Goldschmidt.  The dialectics of sketching , 1991 .

[12]  R. Logie,et al.  Visuo-spatial working memory: Visual, spatial, or central executive? , 1991 .

[13]  Rachel Hewson,et al.  Marking and making : a characterisation of sketching for typographic design , 1995 .

[14]  P. Johnson-Laird,et al.  The mental representation of spatial descriptions , 1982, Memory & cognition.

[15]  S Lammers,et al.  Programmers at Work: Interviews with 19 of Today's Most Brilliant Programmers , 1986 .

[16]  Jonathan Stephen Fish,et al.  Amplifying the Mind’s Eye: Sketching and Visual Cognition , 1990 .

[17]  R A Chechile,et al.  A syntactic complexity effect with visual patterns: evidence for the syntactic nature of the memory representation. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[18]  Kenneth M. Heilman,et al.  The use of hemi-imagery for studying brain asymmetries in image generation , 1996, Neuropsychologia.

[19]  Matthew W. Lewis,et al.  Self-Explonations: How Students Study and Use Examples in Learning to Solve Problems , 1989, Cogn. Sci..

[20]  Daniel L. Schwartz,et al.  Analog Imagery in Mental Model Reasoning: Depictive Models , 1996, Cognitive Psychology.

[21]  Marian Petre,et al.  Why looking isn't always seeing: readership skills and graphical programming , 1995, CACM.

[22]  Alan F. Blackwell,et al.  Metacognitive theories of visual programming: what do we think we are doing? , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[23]  B. Byrne,et al.  Item concreteness vs spatial organization as predictors of visual imagery , 1974, Memory & cognition.

[24]  S M Kosslyn,et al.  Visual images preserve metric spatial information: evidence from studies of image scanning. , 1978, Journal of experimental psychology. Human perception and performance.

[25]  Laurie B. Waisel,et al.  Using Diagrammatic Representations in Mathematical Modeling: The Sketches of Expert Modelers , 1997 .

[26]  R. Dreistadt,et al.  An analysis of the use of analogies and metaphors in science. , 1968, The Journal of psychology.

[27]  Daniel Reisberg,et al.  Can mental images be ambiguous , 1985 .

[28]  D. G. MacKay,et al.  Metaphor and Thought , 1980 .

[29]  Gerald F. Wheeler,et al.  Tough work: Imagery in Scientific thought, creating 20th‐Century physics , 1986 .

[30]  Marian Petre,et al.  Issues Governing the Suitability of Programming Languages for Programming Tasks , 1988, BCS HCI.

[31]  Mark Woodman.  Programming language choice practice and experience , 1996 .

[32]  Rita E. Anderson,et al.  Visual discovery in mind and on paper , 1993, Memory & cognition.

[33]  Gabriela Goldschmidt,et al.  On visual design thinking: the vis kids of architecture , 1994 .

[34]  Stephen J. Payne,et al.  Memory for mental models of spatial descriptions: An episodic-construction-trace hypothesis , 1993, Memory & cognition.

[35]  Pamela Schenk,et al.  The role of drawing in the graphic design process , 1991 .

[36]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[37]  C. H. Ernest.  Imagery ability and cognition: A critical review. , 1977 .

[38]  Yvonne Rogers,et al.  External cognition: how do graphical representations work? , 1996, Int. J. Hum. Comput. Stud..

[39]  Zenon W. Pylyshyn,et al.  What the Mind’s Eye Tells the Mind’s Brain: A Critique of Mental Imagery , 1973 .

[40]  Alan F. Blackwell,et al.  Visual programming: the outlook from academia and industry , 1997, ESP '97.

[41]  D. Levine,et al.  Visual and spatial mental imagery: Dissociable systems of representation , 1988, Cognitive Psychology.

[42]  M Hegarty,et al.  Mental animation in the visuospatial sketchpad: Evidence from dual-task studies , 1997, Memory & cognition.

[43]  Paul Brna,et al.  Supporting the use of external representation in problem solving: the need for flexible learning environments , 1995 .

[44]  Marian Petre,et al.  What experts want from programming languages , 1991 .

[45]  Marian Petre.  Findings a basis for matching programming languages to programming tasks , 1989 .

[46]  Drew H. Abney,et al.  Journal of Experimental Psychology : Human Perception and Performance Influence of Musical Groove on Postural Sway , 2015 .

[47]  John C. Yuille,et al.  Imagery, memory, and cognition : essays in honor of Allan Paivio , 1984 .

[48]  E. Bisiach,et al.  Unilateral Neglect of Representational Space , 1978, Cortex.

[49]  Richard Cox.  Analytical reasoning with multiple external representations , 1996 .

[50]  V. Goel.  Sketches of thought , 1995 .

[51]  Gerald C. Cupchik,et al.  Emerging visions of the aesthetic process : psychology, semiology, and philosophy , 1995 .

[52]  Geir Kaufmann,et al.  Visual Imagery and Its Relation to Problem Solving , 1979 .

[53]  Robert H. Logie,et al.  Mental images in human cognition , 1991 .

[54]  Kevin M. Brooks,et al.  Thoughts beyond words : When language overshadows insight , 1993 .

[55]  S. Hishitani,et al.  Imagery experts: How do expert abacus operators process imagery? , 1990 .

[56]  M. Petre,et al.  Usability Analysis of Visual Programming Environments , 1996 .

[57]  Michel Denis,et al.  Imagery and Thinking , 1991 .

[58]  Steven Pinker,et al.  Reinterpreting Visual Patterns in Mental Imagery , 1989, Cogn. Sci..

[59]  R A Finke,et al.  Explorations of creative visual synthesis in mental imagery , 1988, Memory & cognition.

[60]  L. Brooks.  The Suppression of Visualization by Reading , 1967, The Quarterly journal of experimental psychology.

[61]  Barbara Tversky,et al.  Searching Imagined Environments , 1990 .