Space, Time, and Story

Abstract Life presents as a continuous multimodal barrage on all our senses. From that, we abstract events, discrete units characterized by completion of goals and peaks of action. Effective communication of sequences of events in explanations and narratives is similarly segmented, and linked globally by overall themes and locally by anaphora. Visuospatial explanations and narratives, notably diagrams, comics, and gestures, rely on congruity of mappings of elements and relations of ideas to space and marks in space. Just as we design visuospatial discourse, we design the world: Our design actions in space create diagrams in the world, patterns, piles, rows, one-to-one correspondences, and the like, that express abstractions, categories, hierarchies, dimensions, and more, a circular process termed spraction .

[1]  J. Jones.  Tools for Thought , 1977 .

[2]  Barbara Tversky,et al.  How to put things together , 2012, Cognitive Processing.

[3]  John B. Black,et al.  Coordinating Gesture, Word, and Diagram: Explanations for Experts and Novices , 2015, Spatial Cogn. Comput..

[4]  Barbara Tversky,et al.  Visualizing Thought , 2011, Top. Cogn. Sci..

[5]  Julie Bauer Morrison,et al.  Does animation facilitate learning? An evaluation of the congruence and equivalence hypotheses , 2000 .

[6]  W. Eisner.  Comics and Sequential Art , 1985 .

[7]  Jeffrey M. Zacks,et al.  Perceiving, remembering, and communicating structure in events. , 2001, Journal of experimental psychology. General.

[8]  Barbara Tversky,et al.  Spatial thought, social thought , 2011 .

[9]  H. Simon,et al.  Perception in chess , 1973 .

[10]  Barbara Tversky,et al.  Spinning the Stories of our Lives , 2004 .

[11]  John B. Black,et al.  From Hands to Minds: Gestures Promote Action Understanding , 2012, CogSci.

[12]  J. Bruner.  Actual minds, possible worlds , 1985 .

[13]  Barbara Tversky,et al.  Visualizing space, time, and agents: production, performance, and preference , 2011, Cognitive Processing.

[14]  B. Tversky,et al.  Making sense of abstract events: Building event schemas , 2006, Memory & cognition.

[15]  Barbara Tversky,et al.  Structuring information interfaces for procedural learning. , 2003, Journal of experimental psychology. Applied.

[16]  Karel van der Waarde,et al.  Icons: Support or substitute? , 2000 .

[17]  M. Denis.  The description of routes : A cognitive approach to the production of spatial discourse , 1997 .

[18]  Barbara Tversky,et al.  Arrows in Comprehending and Producing Mechanical Diagrams , 2006, Cogn. Sci..

[19]  Barbara Tversky,et al.  Telling a Story or Telling it Straight: The Effects of Entertaining Versus Accurate Retellings on Memory , 2004 .

[20]  Paul U. Lee,et al.  How Space Structures Language , 1998, Spatial Cognition.

[21]  Masaki Suwa,et al.  Thinking with Sketches , 2009 .

[22]  Y. Anzai.  Learning and use of representations for physics expertise , 1991 .

[23]  Barbara Tversky,et al.  Event Representation in Language and Cognition: Talking about events , 2010 .

[24]  M. Suwa.  Constructive perception in design , 2001 .

[25]  Daniel R. Montello,et al.  Spatial Reasoning With External Visualizations: What Matters Is What You See, Not Whether You Interact , 2008, Cogn. Sci..

[26]  Darren Newtson,et al.  The objective basis of behavior units. , 1977 .

[27]  D. McNeill.  Hand and Mind , 1995 .

[28]  Darren Newtson.  Attribution and the unit of perception of ongoing behavior. , 1973 .

[29]  Yvonne Rogers,et al.  External cognition: how do graphical representations work? , 1996, Int. J. Hum. Comput. Stud..

[30]  Pat Hanrahan,et al.  Identification and validation of cognitive design principles for automated generation of assembly instructions , 2004, AVI.

[31]  D. McNeill.  Gesture and Thought , 2005 .

[32]  B. Tversky,et al.  The shape of action. , 2011, Journal of experimental psychology. General.

[33]  Barbara Tversky,et al.  Animation: can it facilitate? , 2002, Int. J. Hum. Comput. Stud..

[34]  M. Denis,et al.  Spatial discourse and navigation: an analysis of route directions in the city of Venice , 1999 .

[35]  M. Just,et al.  Constructing mental models of machines from text and diagrams. , 1993 .

[36]  Paul U. Lee,et al.  Pictorial and Verbal Tools for Conveying Routes , 1999, COSIT.

[37]  T. Mueller.  Learning to Think Spatially , 2006 .

[38]  Donald A. Norman,et al.  Things that make us smart , 1979 .

[39]  Barbara Tversky,et al.  Explanations in Gesture, Diagram, and Word , 2009, Spatial Language and Dialogue.

[40]  Jeffrey M. Zacks,et al.  Event structure in perception and conception. , 2001, Psychological bulletin.

[41]  Gabriela Goldschmidt,et al.  On visual design thinking: the vis kids of architecture , 1994 .

[42]  Barbara Tversky,et al.  Biased Retellings of Events Yield Biased Memories , 2000, Cognitive Psychology.

[43]  Barbara Tversky,et al.  The Cambridge Handbook of Visuospatial Thinking: Functional Significance of Visuospatial Representations , 2005 .

[44]  Darren Newtson,et al.  The perceptual organization of ongoing behavior , 1976 .

[45]  Chris Stolte,et al.  Rendering effective route maps: improving usability through generalization , 2001, SIGGRAPH.

[46]  Despina A. Stylianou,et al.  On the interaction of visualization and analysis: the negotiation of a visual representation in expert problem solving , 2002 .

[47]  Jeffrey Heer,et al.  Animated Transitions in Statistical Data Graphics , 2007, IEEE Transactions on Visualization and Computer Graphics.

[48]  John R. Anderson,et al.  Abstract Planning and Perceptual Chunks: Elements of Expertise in Geometry , 1990, Cogn. Sci..

[49]  Pat Hanrahan,et al.  Designing effective step-by-step assembly instructions , 2003, ACM Trans. Graph..

[50]  Barbara Tversky,et al.  The structure of experience. , 2008 .

[51]  Masaki Suwa,et al.  Sketches for Design and Design of Sketches , 2003 .

[52]  M. Hayhoe,et al.  Look-ahead fixations: anticipatory eye movements in natural tasks , 2007, Experimental Brain Research.

[53]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[54]  D. Schoen.  The Reflective Practitioner , 1983 .

[55]  S. Goldin-Meadow,et al.  Hearing Gesture: How Our Hands Help Us Think , 2003 .

[56]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[57]  B. Tversky.  Distortions in memory for maps , 1981, Cognitive Psychology.