Real-Time Language Processing as Embodied and Embedded in Joint Action

As the field of psycholinguistics gradually confronts the evidence that language is enmeshed with perceptual-motor processes, it may be in for a shock when it learns that perceptual-motor processes are themselves comprised of the relationship between organism and environment (e.g., ecological perception and active externalism). Therefore, not only is language interactive with perception and action, it is part of the continuous perception-action loop that a person develops with his/her environment. Crucially, one’s environment often has other people whose linguistic and perceptual-motor processes are also being externalised via their perception-action loops. As a result, real-time language processing appears to play a key role in allowing multiple people’s perception-action loops to become entangled and to produce interesting varieties of behavioural synchrony.

[1]  Michael J. Spivey,et al.  Probabilistic Constraint Satisfaction at the Lexical/Phonetic Interface: Evidence for Gradient Effects of Within-Category VOT on Lexical Access , 2003, Journal of psycholinguistic research.

[2]  Michael P. Kaschak,et al.  Temporal Dynamics of the Action–Sentence Compatibility Effect , 2008, Quarterly journal of experimental psychology.

[3]  V. Eguíluz,et al.  Cooperation and the Emergence of Role Differentiation in the Dynamics of Social Networks1 , 2005, American Journal of Sociology.

[4]  Michael J. Spivey,et al.  Linguistically Mediated Visual Search , 2001, Psychological science.

[5]  A. Senghas,et al.  Children Creating Language: How Nicaraguan Sign Language Acquired a Spatial Grammar , 2001, Psychological science.

[6]  Julie C. Sedivy,et al.  Eye movements as a window into real-time spoken language comprehension in natural contexts , 1995, Journal of psycholinguistic research.

[7]  Julie C. Sedivy,et al.  Eye movements to pictures reveal transient semantic activation during spoken word recognition. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[8]  M. G. Wade,et al.  Perceiving Affordances for Aperture Passage in an Environment–Person–Person System , 2009, Journal of motor behavior.

[9]  Gary Lupyan,et al.  The Impact of Communicative Constraints on the Emergence of a Graphical Communication System , 2013, CogSci.

[10]  M. Pickering,et al.  Why is conversation so easy? , 2004, Trends in Cognitive Sciences.

[11]  Janet Beavin Bavelas,et al.  An experimental study of when and how speakers use gestures to communicate , 2002 .

[12]  Bruno Galantucci,et al.  An Experimental Study of the Emergence of Human Communication Systems , 2005, Cogn. Sci..

[13]  Paul P. Maglio,et al.  On Distinguishing Epistemic from Pragmatic Action , 1994, Cogn. Sci..

[14]  H. Bekkering,et al.  Joint action: bodies and minds moving together , 2006, Trends in Cognitive Sciences.

[15]  Bodo Winter,et al.  Processing motion implied in language: eye-movement differences during aspect comprehension , 2012, Cognitive Processing.

[16]  C. Frith,et al.  Language as a Tool for Interacting Minds , 2010 .

[17]  S. McDonald,et al.  Models of high-dimensional semantic space predict language-mediated eye movements in the visual world. , 2006, Acta psychologica.

[18]  Sotaro Kita,et al.  How representational gestures help speaking , 2000 .

[19]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[20]  M. Tanenhaus,et al.  Looking at the rope when looking for the snake: Conceptually mediated eye movements during spoken-word recognition , 2005, Psychonomic bulletin & review.

[21]  Daniel C. Richardson,et al.  Looking To Understand: The Coupling Between Speakers' and Listeners' Eye Movements and Its Relationship to Discourse Comprehension , 2005, Cogn. Sci..

[22]  Rolf A. Zwaan,et al.  Moving words: dynamic representations in language comprehension , 2004, Cognitive Sciences.

[23]  W. Uttal The New Phrenology: The Limits of Localizing Cognitive Processes in the Brain , 2001 .

[24]  Herbert H. Clark,et al.  Grounding in communication , 1991, Perspectives on socially shared cognition.

[25]  U. Neisser Cognition and reality: principles and implications , 1976 .

[26]  C. Frith,et al.  Coming to Terms , 2012, Psychological science.

[27]  Daniel C. Richardson,et al.  Spatial representations activated during real-time comprehension of verbs , 2003, Cogn. Sci..

[28]  J. Gibson The Ecological Approach to Visual Perception , 1979 .

[29]  D. McNeill Gesture and Thought , 2005 .

[30]  J. Austin How to do things with words , 1962 .

[31]  Rolf A. Zwaan,et al.  PSYCHOLOGICAL SCIENCE Research Article THE EFFECT OF IMPLIED ORIENTATION DERIVED FROM VERBAL CONTEXT ON PICTURE RECOGNITION , 2022 .

[32]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[33]  W. Marslen-Wilson Functional parallelism in spoken word-recognition , 1987, Cognition.

[34]  Daniel C. Richardson,et al.  The Art of Conversation Is Coordination , 2007, Psychological science.

[35]  Herbert H. Clark,et al.  Contributing to Discourse , 1989, Cogn. Sci..

[36]  P. Deltenre,et al.  Mismatch negativity evoked by the McGurk–MacDonald effect: a phonetic representation within short-term memory , 2002, Clinical Neurophysiology.

[37]  Michael L. Anderson,et al.  Investigating Functional Cooperation in the Human Brain Using Simple Graph-Theoretic Methods , 2010 .

[38]  J. S. Jordan,et al.  Learned patterns of action-effect anticipation contribute to the spatial displacement of continuously moving stimuli. , 2008, Journal of experimental psychology. Human perception and performance.

[39]  J. Fodor,et al.  The Modularity of Mind: An Essay on Faculty Psychology , 1984 .

[40]  Robert L. Goldstone,et al.  Adaptive Group Coordination , 2009 .

[41]  M Corbetta,et al.  Frontoparietal cortical networks for directing attention and the eye to visual locations: identical, independent, or overlapping neural systems? , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[42]  Michael J. Richardson,et al.  Judging and actualizing intrapersonal and interpersonal affordances. , 2007, Journal of experimental psychology. Human perception and performance.

[43]  M. Tanenhaus,et al.  Actions and affordances in syntactic ambiguity resolution. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[44]  A. Koller,et al.  Speech Acts: An Essay in the Philosophy of Language , 1969 .

[45]  Martha W. Alibali,et al.  Gesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information , 2005, Spatial Cogn. Comput..

[46]  S. Read,et al.  Virtual Personalities: A Neural Network Model of Personality , 2007 .

[47]  Daniel C. Richardson,et al.  Conversation and Coordinative Structures , 2009, Top. Cogn. Sci..

[48]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[49]  Michael J. Spivey,et al.  Inefficient conjunction search made efficient by concurrent spoken delivery of target identity , 2006, Perception & psychophysics.

[50]  V. Druskat,et al.  Building the emotional intelligence of groups. , 2001, Harvard business review.

[51]  M. Natale CONVERGENCE OF MEAN VOCAL INTENSITY IN DYADIC COMMUNICATION AS A FUNCTION OF SOCIAL DESIRABILITY , 1975 .

[52]  M. Turvey Affordances and Prospective Control: An Outline of the Ontology , 1992 .

[53]  Michael J. Richardson,et al.  Affording cooperation: Embodied constraints, dynamics, and action-scaled invariance in joint lifting , 2010, Psychonomic bulletin & review.

[54]  D. Ballard,et al.  Memory Representations in Natural Tasks , 1995, Journal of Cognitive Neuroscience.

[55]  Holly Arrow,et al.  Stability, Bistability, and Instability in Small Group Influence Patterns , 1997 .

[56]  Michael J. Spivey,et al.  Oculomotor mechanisms activated by imagery and memory: eye movements to absent objects , 2001, Psychological research.

[57]  R. Ilmoniemi,et al.  Functional links between motor and language systems , 2005, The European journal of neuroscience.

[58]  Olivia De La Rocha,et al.  The Reorganization of Arithmetic Practice in the Kitchen. , 1985 .

[59]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[60]  H. Deubel,et al.  Saccade target selection and object recognition: Evidence for a common attentional mechanism , 1996, Vision Research.

[61]  Marc A. Smith Voices from the WELL: The Logic of the Virtual Commons , 1992 .

[62]  N. Ambady,et al.  A dynamic interactive theory of person construal. , 2011, Psychological review.

[63]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[64]  A. Fuchs,et al.  Coordination: Neural, Behavioral and Social Dynamics , 2008 .

[65]  M. Jeannerod,et al.  Language-Induced Motor Perturbations during the Execution of a Reaching Movement , 2008, Quarterly journal of experimental psychology.

[66]  Howard GCiles ACCENT MOBILITY: A MODEL AND SOME DATA , 2016 .

[67]  Robert L. Goldstone,et al.  Author ' s personal copy Recognizing group cognition Action editor : , 2010 .

[68]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[69]  Bahador Bahrami,et al.  Visual motion interferes with lexical decision on motion words , 2008, Current Biology.

[70]  Julie C. Sedivy,et al.  Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution , 2002, Cognitive Psychology.

[71]  R. Sternberg,et al.  Group intelligence: Why some groups are better than others , 1988 .

[72]  L Frazier,et al.  Constraint satisfaction as a theory of sentence processing , 1995, Journal of psycholinguistic research.

[73]  Deborah Tollefsen,et al.  From extended mind to collective mind , 2006, Cognitive Systems Research.

[74]  Paul D. Allopenna,et al.  Tracking the Time Course of Spoken Word Recognition Using Eye Movements: Evidence for Continuous Mapping Models , 1998 .

[75]  Lawrence W. Barsalou,et al.  Perceptions of perceptual symbols , 1999, Behavioral and Brain Sciences.

[76]  Marleen B. Schippers,et al.  Mapping the information flow from one brain to another during gestural communication , 2010, Proceedings of the National Academy of Sciences.

[77]  Riccardo Fusaroli,et al.  The dialogically extended mind: Language as skilful intersubjective engagement , 2014, Cognitive Systems Research.

[78]  David Slepian,et al.  Key papers in the development of information theory , 1974 .

[79]  S. Goldin-Meadow,et al.  The role of gesture in communication and thinking , 1999, Trends in Cognitive Sciences.

[80]  Murat Aydede,et al.  A short primer on situated cognition , 2008 .

[81]  Michael J. Spivey,et al.  Gradiency and Visual Context in Syntactic Garden-Paths. , 2007, Journal of memory and language.

[82]  J. Cappella,et al.  Talk and Silence Sequences in Informal Conversations III: Interspeaker Influence , 1981 .

[83]  Daniel C. Richardson,et al.  Eye movements in language and cognition , 2007 .

[84]  Michael J. Spivey,et al.  The Role of Preview and Incremental Delivery on Visual Search , 2012, CogSci.

[85]  Michael J. Spivey,et al.  Eye Movements and Problem Solving , 2003, Psychological science.

[86]  Michael J. Richardson,et al.  Toward a radically embodied, embedded social psychology , 2009 .

[87]  R. Street Speech Convergence and Speech Evaluation in Fact-Finding Interviews , 1984 .

[88]  John-Dylan Haynes,et al.  Content-specific coordination of listeners' to speakers' EEG during communication , 2012, Front. Hum. Neurosci..

[89]  Maja J. Matarić,et al.  Designing emergent behaviors: from local interactions to collective intelligence , 1993 .

[90]  G. Knoblich,et al.  Action coordination in groups and individuals: learning anticipatory control. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[91]  P. Fusaroli,et al.  Scientific Publications in Endoscopic Ultrasonography: A 20-Year Global Survey of the Literature , 2002, Endoscopy.

[92]  A. Clark Supersizing the Mind , 2008 .

[93]  J. Gold,et al.  Representation of a perceptual decision in developing oculomotor commands , 2000, Nature.

[94]  Michael J. Richardson,et al.  Articulatory constraints on interpersonal postural coordination. , 2007, Journal of experimental psychology. Human perception and performance.

[95]  K. Duncker,et al.  On problem-solving , 1945 .