Response direction and sentence-tense compatibility effects: An eye tracking study Raymond B. Becker (rbecker@cit-ec.uni-bielefeld.de) Bridgette DeCot (bdecot@cit-ec.uni-bielefeld.de) Ernesto Guerra (ernesto.guerra@uni-bielefeld.de) Pia Knoeferle (knoeferl@cit-ec.uni-bielefeld.de) Cognitive Interaction Technology Excellence Cluster and Department of Linguistics, Bielefeld University, Morgenbreede 39, 33615, Bielefeld, Germany Rolf Zwaan (zwaan@fsw.eur.nl) Department of Psychology, Erasmus University Burgemeester Oudlaan 50, 3062 PA, Rotterdam, The Netherlands Abstract Recent evidence shows tense-response compatibility effects only when the task relates to sentence tense (Ulrich & Maienborn, 2010). In two eye-tracking experiments, we investigated tense-response compatibility effects. In our first experiment (E1, where sentence tense was relevant to the task) we found compatibility effects at the beginning of the sentence (e.g., Yesterday versus Tomorrow), which shifted to interference effects by sentence end. Overall, we also found compatibility effects in response times, replicating Ulrich and Maienborn. Both compatibility effects in Experiment 1 (E1) were stronger for low- compared to high-WM readers. In Experiment 2 (E2, where tense was irrelevant), we found compatibility effects for high-WM readers, but only in early reading measures. These results suggest that compatibility effects are weaker depending on the task, but not eliminated; an implication which may help refine a strict view of embodied cognition. Keywords: Mental timeline, differences, eye-tracking. embodiment, individual Introduction Research over the last decade has continued to refine embodiment theory (Barsalou, 1999; Glenberg, 1997), and this refinement was prodded along by criticism (Machery, 2007; Mahon & Caramazza, 2008). For example, Mahon & Caramazza argued that embodiment theory could not adequately explain how JUSTICE and other abstract concepts are understood through bodily experience because they do not reliably correspond to sensory or motor information. However, conceptual metaphor theory has laid out the groundwork for how abstract concepts such as TIME are mapped onto concrete concepts such as SPACE (Lakoff & Johnson, 1980; 1999). Torralbo, Santiago, & Lupianez (2006) found evidence that corroborated this potential mapping mechanism. In their Experiment 1, participants saw the silhouette of a human head looking either rightward or leftward on a screen. A word with a temporal connotation in a speech bubble was presented either in front of or behind the silhouette. Participants judged whether the person represented via the silhouette was contemplating the past or the future. When a past word appeared on the left, responses were faster than when it appeared to the right; when a future word appeared on the right side, responses were faster than when it appeared to the left (this interaction of response- location with tense has been credited to a ‘mental timeline’, i.e., the use of a spatial left-right line to represent time in our mind). These results suggest that left- and right-hand response preparation interacts with linguistic temporal cues (past and future tense respectively). Thus it appears that abstract concepts such as TIME are grounded in experiential and bodily schemas. Meanwhile, the focus of inquiry in this area has changed from whether grounding effects occur for abstract concepts to how rapidly they occur and whether they are task-dependent. In addition, the role of participants’ working memory in these kinds of congruence effects is unclear. To contribute to these research questions we examined the time course of time- response location congruence effects during sentence comprehension as (low and high working memory) participants planned a right or left hand movement in two different tasks. Below we motivate in more detail the investigation of tense-response location congruence effects are modulated by task and working memory. Accommodating tense-response-location congruence effects Task appears to play an important role for tense-response location congruence effects. In a recent study, compatibility effects of tense (e.g., past versus future) and left/right response locations were eliminated in a task where tense was irrelevant. When participants paid attention to sentence tense, tense-response location compatibility effects emerged. For example, participants pressed a button labeled Past on the left in response to a past tense sentence more quickly than when the Past button was on the right. A similar compatibility effect was found for future-tense sentences and right-hand responses. However, when the task was time-irrelevant (sentence-sensibility judgments), compatibility effects were eliminated, suggesting that time- response location compatibility effects occur only when people pay attention to time. If that were the case, then both embodied (e.g., Barsalou, 1999) and non-embodied accounts such as amodal symbol systems (Collins & Loftus, 1975, Collins & Quillian, 1969) could accommodate these results. Non-embodied accounts could accommodate the
[1]
Michael J. Spivey,et al.
The Continuity Of Mind
,
2008
.
[2]
Robert J Hartsuiker,et al.
Looking, language, and memory: bridging research from the visual world and visual search paradigms.
,
2011,
Acta psychologica.
[3]
Michael J. Spivey,et al.
Oculomotor mechanisms activated by imagery and memory: eye movements to absent objects
,
2001,
Psychological research.
[4]
Richard L. Lewis,et al.
Computational principles of working memory in sentence comprehension
,
2006,
Trends in Cognitive Sciences.
[5]
Richard P. Heitz,et al.
Complex working memory span tasks and higher-order cognition: A latent-variable analysis of the relationship between processing and storage
,
2009,
Memory.
[6]
R. Baayen,et al.
Mixed-effects modeling with crossed random effects for subjects and items
,
2008
.
[7]
Deanna M. Barch,et al.
Working Memory Related Brain Network Connectivity in Individuals with Schizophrenia and Their Siblings
,
2012,
Front. Hum. Neurosci..
[8]
D. Kemmerer,et al.
The Two-Level Theory of verb meaning: An approach to integrating the semantics of action with the mirror neuron system
,
2010,
Brain and Language.
[9]
R. Ulrich,et al.
Left–right coding of past and future in language: The mental timeline during sentence processing
,
2010,
Cognition.
[10]
A. Glenberg,et al.
Action-based language: A theory of language acquisition, comprehension, and production
,
2012,
Cortex.
[11]
Allan Collins,et al.
A spreading-activation theory of semantic processing
,
1975
.
[12]
Marta Kutas,et al.
When temporal terms belie conceptual order
,
1998,
Nature.
[13]
G. Lakoff,et al.
Metaphors We Live by
,
1982
.
[14]
M. Ross Quillian,et al.
Retrieval time from semantic memory
,
1969
.
[15]
Bradford Z. Mahon,et al.
A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content
,
2008,
Journal of Physiology-Paris.
[16]
Ed Symes,et al.
Dissociating object-based and space-based affordances
,
2005
.
[17]
A. Clark.
Being There: Putting Brain, Body, and World Together Again
,
1996
.
[18]
D. Barr,et al.
Random effects structure for confirmatory hypothesis testing: Keep it maximal.
,
2013,
Journal of memory and language.
[19]
A. Glenberg,et al.
Grounding language in bodily states: The case for emotion
,
2005
.
[20]
A. Baddeley.
Working memory: theories, models, and controversies.
,
2012,
Annual review of psychology.
[21]
Julio Santiago,et al.
Flexible Conceptual Projection of Time Onto Spatial Frames of Reference
,
2006,
Cogn. Sci..
[22]
G. Lakoff.
Philosophy in the flesh
,
1999
.
[23]
Lawrence W. Barsalou,et al.
Perceptions of perceptual symbols
,
1999,
Behavioral and Brain Sciences.
[24]
Edouard Machery,et al.
Concept empiricism: A methodological critique
,
2007,
Cognition.
[25]
Patrick Jeuniaux,et al.
Language comprehension is both embodied and symbolic
,
2008
.
[26]
Dana Chidekel,et al.
From Movement to Thought: Executive Function, Embodied Cognition, and the Cerebellum
,
2012,
The Cerebellum.