Digitizing Literacy: Reflections on the Haptics of Writing

Writing is a complex cognitive process relying on intricate perceptual-sensorimotor combinations. The process and skill of writing is studied on several levels and in many disciplines, from neurophysiological research on the shaping of each letter to studies on stylistic and compositional features of authors and poets. In studies of writing and literacy overall, the role of the physically tangible writing device (pen on paper; computer mouse and keyboard; digital stylus pen and writing tablet; etc.) is rarely addressed. By and large, the (relatively young) field of writing research is dominated by cognitive approaches predominantly focusing on the visual component of the writing process, hence maintaining a separation between (visual) perception and motor action (e.g., haptics1). However, recent theoretical currents in psychology, phenomenology & philosophy of mind, and neuroscience – commonly referred to as “embodied cognition” – indicate that perception and motor action are closely connected and, indeed, reciprocally dependent. Today, most of our writing is done with digital writing devices (the computer, the mobile phone, the PDA [i.e., Personal Digital Assistant]), rather than writing by hand. The switch from pen and paper to mouse, keyboard and screen entails major differences in the haptics of writing, at several distinct but intersecting levels. Handwriting is by essence a unimanual activity, whereas typewriting is bimanual. Typically, handwriting is also a slower process than typewriting. Moreover, the visual attention of the writer is strongly concentrated during handwriting; the attentional focus of the writer is dedicated to the tip of the pen, while during typewriting the visual attention is detached from the haptic input, namely the

[1]  Jerome L. Singer,et al.  Imagination and Play in the Electronic Age , 2005 .

[2]  Jean-Luc Velay,et al.  Visual presentation of single letters activates a premotor area involved in writing , 2003, NeuroImage.

[3]  D. Gentner The acquisition of typewriting skill , 1983 .

[4]  E. Rosch,et al.  The Embodied Mind: Cognitive Science and Human Experience , 1993 .

[5]  M. Roth,et al.  Premotor activations in response to visually presented single letters depend on the hand used to write: a study on left-handers , 2005, Neuropsychologia.

[6]  Gerard P. van Galen,et al.  Handwriting: Issues for a psychomotor theory ☆ , 1991 .

[7]  C. Gonnella The Working Brain: An Introduction to Neuropsychology , 1976 .

[8]  Gunther Kress,et al.  Literacy in the New Media Age , 2003 .

[9]  J. Gibson The Ecological Approach to Visual Perception , 1979 .

[10]  C. Spence,et al.  The Handbook of Multisensory Processing , 2004 .

[11]  William D. Taylor,et al.  Orality and literacy: The technologizing of the word , 1984 .

[12]  A. Luria The Working Brain: An Introduction To Neuropsychology , 1976 .

[13]  Alexander Refsum Jensenius,et al.  Action-sound : developing methods and tools to study music-related body movement , 2007 .

[14]  C. Kato,et al.  Involvement of motor cortices in retrieval of kanji studied by functional MRI. , 1999, Neuroreport.

[15]  Panagiota Alevizou,et al.  Beyond Technology: Children's Learning in the Age of Digital Culture , 2007 .

[16]  J. Gilhodes,et al.  Remembering the orientation of newly learned characters depends on the associated writing knowledge: a comparison between handwriting and typing. , 2006, Human movement science.

[17]  B. O'Shaughnessy,et al.  Consciousness And The World , 2000 .

[18]  Alex Martin,et al.  Representation of Manipulable Man-Made Objects in the Dorsal Stream , 2000, NeuroImage.

[19]  Charles A. MacArthur,et al.  Handbook of Writing Research , 2005 .

[20]  Maurice Merleau-Ponty Phenomenology of Perception , 1964 .

[21]  Susan J. Lederman,et al.  Visual and haptic exploratory procedures in children's judgments about tool function , 2005 .

[22]  A. Vinter,et al.  Visual and proprioceptive recognition of cursive letters in young children. , 2008, Acta psychologica.

[23]  Mark Torrance,et al.  Writing and cognition: research and applications , 2007 .

[24]  W. Ong,et al.  Orality and literacy : the technologizing of the word , 1982 .

[25]  J. Waller,et al.  Mind in Life: Biology, Phenomenology, and the Sciences of Mind , 2009 .

[26]  G. Glover,et al.  Finger movements lighten neural loads in the recognition of ideographic characters. , 2003, Brain research. Cognitive brain research.

[27]  P. Denes On the Motor Theory of Speech Perception , 1965 .

[28]  Faith Kurtyka,et al.  Literacy: An Introduction to the Ecology of Written Language 2nd ed (review) , 2012 .

[29]  B. Schlaggar,et al.  Book Review Proust and the Squid: The Story and Science of the Reading Brain By Maryanne Wolf. 308 pp., illustrated. New York, Harper, 2007. $25.95. 978-0-06-018639-5 , 2008 .

[30]  Christine M. Neuwirth,et al.  Writing and digital media , 2006 .

[31]  David Buckingham,et al.  Media Education: Literacy, Learning and Contemporary Culture , 2003 .

[32]  E. Thompson,et al.  Mind in life : biology, phenomenology, and the sciences of mind , 2007 .

[33]  F. R. Wilson The Hand: How Its Use Shapes the Brain, Language, and Human Culture , 1998 .

[34]  T. Shanahan,et al.  Reading and Writing Relations and Their Development , 2000 .

[35]  S. Goldin-Meadow,et al.  Hearing Gesture: How Our Hands Help Us Think , 2003 .

[36]  Michael Henry Heim,et al.  Electric Language: A Philosophical Study of Word Processing , 1987 .

[37]  W. Benjamin The work of art in the age of mechanical reproduction , 2018, A Museum Studies Approach to Heritage.

[38]  Brockenbrough S. Allen,et al.  MEDIA AS LIVED ENVIRONMENTS: THE ECOLOGICAL PSYCHOLOGY OF EDUCATIONAL TECHNOLOGY , 1996 .

[39]  Friedemann Pulvermüller,et al.  Brain mechanisms linking language and action , 2005, Nature Reviews Neuroscience.

[40]  Colin Lankshear,et al.  New Literacies: Everyday Practices and Classroom Learning , 2006 .

[41]  G. Olivier,et al.  Visual objects can potentiate a grasping neural simulation which interferes with manual response execution. , 2009, Acta psychologica.

[42]  K. James,et al.  Letter processing automatically recruits a sensory–motor brain network , 2006, Neuropsychologia.

[43]  Frank A. Logan,et al.  Errors in copy typewriting , 1999 .

[44]  Carey Jewitt,et al.  Technology, Literacy, Learning: A Multimodal Approach , 2005 .

[45]  C. Hulme The Interaction of Visual and Motor Memory for Graphic Forms following Tracing , 1979, The Quarterly journal of experimental psychology.

[46]  Jean-Claude Gilhodes,et al.  Learning through Hand- or Typewriting Influences Visual Recognition of New Graphic Shapes: Behavioral and Functional Imaging Evidence , 2008, Journal of Cognitive Neuroscience.

[47]  M. Naka,et al.  The effect of repeated writing on memory , 1995, Memory & cognition.

[48]  Liora Bresler,et al.  Fifty Major Thinkers on Education: From Confucius to Dewey , 2002 .

[49]  Sharon Vaughn,et al.  Early Spelling Acquisition: Does Writing Really Beat the Computer? , 1992 .

[50]  Alva Noë,et al.  Action in Perception , 2006, Representation and Mind.

[51]  Keith E. Stanovich,et al.  Early Spelling Acquisition: Writing Beats the Computer , 1990 .

[52]  David Barton,et al.  Literacy: An Introduction to the Ecology of Written Language , 1994 .

[53]  Roz Ivanic,et al.  Situated literacies : reading and writing in context , 2000 .

[54]  H. Zettl Sight, Sound, Motion: Applied Media Aesthetics , 1973 .

[55]  P. Greenfield,et al.  Language, tools and brain: The ontogeny and phylogeny of hierarchically organized sequential behavior , 1991, Behavioral and Brain Sciences.

[56]  Christina Haas,et al.  Writing Technology: Studies on the Materiality of Literacy , 1995 .

[57]  J. Velay,et al.  The influence of writing practice on letter recognition in preschool children: a comparison between handwriting and typing. , 2005, Acta psychologica.

[58]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[59]  David H. Jonassen,et al.  Handbook of Research for educational Communications and Technology , 1997 .

[60]  Jay David Bolter,et al.  Writing Space: Computers, HyperText, and the Remediation of Print , 2001 .

[61]  Pascale Colé,et al.  Haptics in learning to read with children from low socio-economic status families , 2007 .

[62]  Carey Jewitt,et al.  Technology, literacy and learning: a multimodal approach , 2010 .

[63]  R. Klatzky,et al.  Haptic exploration in the presence of vision. , 1993, Journal of experimental psychology. Human perception and performance.

[64]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.