Gestures Alter Thinking About Time Azadeh Jamalian (aj2334@columbia.edu) Teachers College, Columbia University, 525 W. 120 th Street New York, NY 10027 USA Barbara Tversky (btversky@stanford.edu) Teachers College, Columbia University, 525 W. 120 th Street New York, NY 10027 USA another, “time has passed”, and “the future is ahead of us” (e. g., Clark, 1973; Evans, 2003; Lakoff and Johnson, 1999, Moore, 2006; Nunez, 1999). People’s diagrams of events in time, such as the meals of a day, are typically ordered in reading order on a horizontal line (Tversky, Kugelmass, & Winter, 1991). When relating events in time, English speakers often move their hands from left to right, event by event (e.g. Cienki, 1998); they point frontwards for the future and backwards for the past (e.g. Cooperrider & Nunez, 2009). Language, diagrams, and gestures are ways of externalizing thought, and are congruent with thinking (Tversky, 2011). If people use actions in space to express their conceptions of events in time, will seeing different forms of actions in space change their understanding of time? We address this question here, by explaining temporal events with identical language but different gestures. Speakers everywhere gesture while they speak. Most gestures are redundant with the speech they accompany (McNeill, 1992), but gestures sometimes express information that is not expressed in speech (e. g., McNeil, 1992; Church & Goldin-Meadow, 1986; Goldin-Meadow, Alibali, & Church, 1993; Perry, Church, & Goldin-Meadow, 1988). Although some have questioned the communicative significance of gestures (Krauss, 1998; Rauscher, Krauss, Chen, 1996; Rime & Shiaratura, 1991), there is good evidence that speakers often intend their gestures to be communicative (e. g., Cohen, 1977; Cohen & Harrison, 1973; Alibali, Heat, & Myers, 2001; Emmorey & Casey, 2001), and that gestures, whether redundant or mismatching, influence addressees’ comprehension (Goldin-Meadow & Sandhofer, 1999; Thompson & Massaro, 1994). Can the unique information in gesture alter listeners’ mental models of a highly abstract yet familiar concept? In a series of studies on reasoning about time, we demonstrate that gestures affect addressee’s conceptions of time by keeping speech constant but altering gestures. Abstract Can gestures alter thought? Thinking about time is deeply interlinked with actions in space, and gestures are abstracted actions. Four experiments showed that gestures alter thinking about time. Participants heard the same speech accompanied by different gestures. The viewed gestures biased listeners toward circular or linear thinking, toward parallel or sequential thinking, toward moving-ego or moving-time perspectives. Gestures can abstract and show mental models more directly and succinctly than speech. Keywords: Gesture, space, time, metaphor, diagram Introduction As they say, life is just one thing after another. But there is more complexity to thinking about events in time. Historical and autobiographical events are often regarded as on a timeline, but events can also happen simultaneously, not a simple single sequence. Repeating events like seasons, days, and the cell cycle can be regarded as circular. Moreover, reasoning about events in time entails taking a perspective on the timeline. Two common perspectives are moving-ego, thinking of yourself as moving along a timeline (we’re approaching summer), or moving-time, thinking of yourself as stationary on a timeline with events moving past you (summer is approaching) (e. g., Clark, 1973). These perspectives are analogous to a route or intrinsic or egocentric perspective in space; the viewpoint is embedded in space or in time, with ego as the reference (e.g., Levinson, 1996; Tversky, 1996). But just as it is possible to take an external or survey or absolute perspective on space, it is possible to take an external or absolute or calendar view on time, an outside perspective regarding events as ordered by dates. In the case of survey/absolute spatial perspective, the reference points are landmarks and the terms of reference are typically north-south-east-west. For external/absolute/calendar temporal perspective, the reference points are dates or events, and the terms of reference are earlier/later. Whatever the perspective, how people think about events in time is highly interlinked to actions in space (Talmy, 2000; Tversky, 2011). The strong association between action, space, and time is reflected in the language people use when talking about time, the diagrams they draw when conveying events in time, and the gestures that accompany narratives of events in time. People say time “marches on”, we “move through” time, one event occurs “before” 1: Circular vs. Linear Thinking: Diagram Prior work (Kessell & Tversky, submitted) has shown that people are biased towards linear thinking. Participants were asked to diagram four-step cyclical or sequential processes. Most participants drew linear diagrams even for cycles. Expecting congruency between conception and visualization, Kessell and Tversky concluded that circular
[1]
K. Emmorey,et al.
Gesture, Thought, and Spatial Language
,
2001
.
[2]
Vyvyan Evans,et al.
The Structure of Time
,
2003
.
[3]
H. Clark,et al.
In cognitive development and the acquisition of language
,
1973
.
[4]
Barbara Tversky,et al.
Visualizing Thought
,
2011,
Top. Cogn. Sci..
[5]
F. Moore.
Cognitive development and the acquisition of language
,
1973
.
[6]
R. B. Church,et al.
A comparison between children's and adults' ability to detect conceptual information conveyed through representational gestures.
,
1998,
Child development.
[7]
L A Thompson,et al.
Children's integration of speech and pointing gestures in comprehension.
,
1994,
Journal of experimental child psychology.
[8]
M. McGlone,et al.
Back (or forward?) to the future: The role of perspective in temporal language comprehension.
,
1998
.
[9]
W. Freeman,et al.
Reclaiming cognition : the primacy of action, intention and emotion
,
1999
.
[10]
M. Alibali,et al.
Transitions in concept acquisition: using the hand to read the mind.
,
1993,
Psychological review.
[11]
A. Cohen,et al.
Intentionality in the use of hand illustrators in face-to-face communication situations.
,
1973
.
[12]
M. Studdert-Kennedy.
Hand and Mind: What Gestures Reveal About Thought.
,
1994
.
[13]
R. Krauss,et al.
Gesture, Speech, and Lexical Access: The Role of Lexical Movements in Speech Production
,
1996
.
[14]
Jean-Pierre Koenig,et al.
Metaphoric gestures and some of their relations to verbal metaphoric expressions
,
1998
.
[15]
Barbara Tversky,et al.
Explanations in Gesture, Diagram, and Word
,
2009,
Spatial Language and Dialogue.
[16]
Susan Goldin-Meadow,et al.
Transitional knowledge in the acquisition of concepts
,
1988
.
[17]
A. Leroi‐Gourhan,et al.
Gesture and Speech
,
1993
.
[18]
Akiba A. Cohen,et al.
The Communicative Functions of Hand I1lustrators
,
1977
.
[19]
S. Levinson.
Frames of reference and Molyneux's question: Cross-linguistic evidence
,
1996
.
[20]
Lera Boroditsky,et al.
The Roles of Body and Mind in Abstract Thought
,
2002,
Psychological science.
[21]
Kensy Cooperrider,et al.
Across time , across the body Transversal temporal gestures
,
2009
.
[22]
G. Lakoff,et al.
Metaphors We Live by
,
1982
.
[23]
A. Glenberg,et al.
Comprehension of illustrated text: Pictures help to build mental models☆
,
1992
.
[24]
T. Chapman,et al.
The Unreality of Time
,
1980
.
[25]
W. H. Newton-Smith,et al.
The Structure of Time
,
2018
.
[26]
B. Tversky,et al.
Cross-cultural and developmental trends in graphic productions
,
1991,
Cognitive Psychology.
[27]
R. Krauss.
Why Do We Gesture When We Speak?
,
1998
.
[28]
L. Boroditsky.
Metaphoric structuring: understanding time through spatial metaphors
,
2000,
Cognition.
[29]
M. Alibali,et al.
Effects of Visibility between Speaker and Listener on Gesture Production: Some Gestures Are Meant to Be Seen
,
2001
.
[30]
Barbara Tversky,et al.
Spatial Perspective in Descriptions
,
1996
.
[31]
S. Levinson,et al.
LANGUAGE AND SPACE
,
1996
.
[32]
P. Johnson-Laird,et al.
How Diagrams Can Improve Reasoning
,
1993
.
[33]
K. E. Moore,et al.
Space-to-time mappings and temporal concepts
,
2006
.
[34]
S. Goldin-Meadow,et al.
The mismatch between gesture and speech as an index of transitional knowledge
,
1986,
Cognition.