A review on complementary natures of tangible user interfaces (TUIs) and early spatial learning

Abstract Spatial skills are essential for everyday tasks, and technology blends seamlessly into children’s everyday environment. Since spatiality as a term is ubiquitous in experience this paper bridges literature in two fields: theories on early spatial learning in cognitive development and potential benefits of tangible user interfaces (TUIs) for supporting very young children’s spatial skills. Studies suggest that the period between 2 and 4 years of age is critical for training spatial skills (e.g., mental rotation), which relate to further success in STEAM (science, technology, engineering, arts, and math) disciplines. We first present a review of the empirical findings on spatial skills, early interventions, and tools (i.e., narrative and gesture input) recommended for training preschool children’s spatial skills. By situating the work within the use and benefits of manipulatives (e.g., building blocks, puzzles, shapes) combined with digital affordances in interaction design, we address the relevance of TUIs as complementary tools for spatial learning. We concentrate on the supporting properties of TUIs that enable playful learning, make storytelling more concrete, and provide embodiment effects through physicality. Through various products found in the market and literature that address the physical–digital convergence, we invite designers and researchers to consider design practices and applicable technology that build on present efforts and paradigms in this area. To contribute to this area, we conclude with a discussion of the gaps in design methods to develop technologies for children younger than 4 years old, and propose directions for future work to leverage new tools that serve very young children’s spatial learning and possible inquiries for dual payoff.

[1]  Denton J. Snider,et al.  The Psychology of Froebel's Play-Gifts , 2010 .

[2]  Timothy S. McNerney From turtles to Tangible Programming Bricks: explorations in physical language design , 2004, Personal and Ubiquitous Computing.

[3]  Leah Buechley,et al.  Boda blocks: a collaborative tool for exploring tangible three-dimensional cellular automata , 2007, CSCL.

[4]  Mitchel Resnick,et al.  Extending tangible interfaces for education: digital montessori-inspired manipulatives , 2005, CHI.

[5]  J. H. Steiger,et al.  Accomplishment in Science, Technology, Engineering, and Mathematics (STEM) and Its Relation to STEM Educational Dose: A 25-Year Longitudinal Study , 2010 .

[6]  Michael Pope,et al.  RoyoBlocks: an exploration in tangible literacy learning , 2013, IDC.

[7]  Lori L. Scarlatos TICLE: using multimedia multimodal guidance to enhance learning , 2002, Inf. Sci..

[8]  Justine Cassell,et al.  StoryMat: a play space for collaborative storytelling , 1999, CHI Extended Abstracts.

[9]  William Buxton,et al.  Graspable user interfaces , 1996 .

[10]  Hiroshi Ishii,et al.  curlybot: designing a new class of computational toys , 2000, CHI.

[11]  D. Lubinski,et al.  Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. , 2009 .

[12]  Hiroshi Ishii,et al.  Jabberstamp: embedding sound and voice in traditional drawings , 2007, SIGGRAPH 2007.

[13]  William H. Teale,et al.  Picture Books and the Digital World , 2014 .

[14]  Mitchel Resnick,et al.  Turtles, termites, and traffic jams - explorations in massively parallel microworlds , 1994 .

[15]  Kathy Hirsh-Pasek,et al.  Putting Education in “Educational” Apps , 2015, Psychological science in the public interest : a journal of the American Psychological Society.

[16]  Mitchel Resnick,et al.  Constructionism in Practice: Designing, Thinking, and Learning in A Digital World , 1996 .

[17]  S. Goldin-Meadow,et al.  Gesturing Gives Children New Ideas About Math , 2009, Psychological science.

[18]  Paul Marshall,et al.  Do tangible interfaces enhance learning? , 2007, TEI.

[19]  Allison Druin,et al.  Cooperative inquiry: developing new technologies for children with children , 1999, CHI '99.

[20]  Caroline Hummels,et al.  The Development of LinguaBytes: An Interactive Tangible Play and Learning System to Stimulate the Language Development of Toddlers with Multiple Disabilities , 2008, Adv. Hum. Comput. Interact..

[21]  Stella F. Lourenco,et al.  Spatial Processing in Infancy Predicts Both Spatial and Mathematical Aptitude in Childhood , 2016, Psychological science.

[22]  Ilgım Veryeri Alaca Materiality in picturebooks , 2017 .

[23]  Kathy Hirsh-Pasek,et al.  Geometric Toys in the Attic? A Corpus Analysis of Early Exposure to Geometric Shapes. , 2016 .

[24]  J. G. Tanenbaum,et al.  The Reading Glove: designing interactions for object-based tangible storytelling , 2010, AH.

[25]  Wendy E. Mackay,et al.  Tangicam: exploring observation tools for children , 2005, IDC '05.

[26]  Mitchel Resnick,et al.  Digital manipulatives: new toys to think with , 1998, CHI.

[27]  Joost Elffers,et al.  Tangram: The Ancient Chinese Shapes Game , 1977 .

[28]  Alissa Nicole Antle,et al.  The CTI framework: informing the design of tangible systems for children , 2007, TEI.

[29]  B. Casey,et al.  Use of a Storytelling Context to Improve Girls' and Boys' Geometry Skills in Kindergarten. , 2008 .

[30]  Allison Druin,et al.  Cooperative Inquiry revisited: Reflections of the past and guidelines for the future of intergenerational co-design , 2013, Int. J. Child Comput. Interact..

[31]  Antonio Rizzo,et al.  Building Narrative Experiences for Children Through Real Time Media Manipulation: Pogo World , 2005, Funology.

[32]  Marc Langheinrich,et al.  Towards guidelines for designing augmented toy environments , 2008, DIS '08.

[33]  Mike Ananny,et al.  Supporting children's collaborative authoring: practicing written literacy while composing oral texts , 2002, CSCL.

[34]  Yasmin B. Kafai,et al.  Fröbel's forgotten gift: textile construction kits as pathways into play, design and computation , 2010, IDC.

[35]  N. Mahajan,et al.  Once upon a Time: Parent-Child Dialogue and Storybook Reading in the Electronic Era. , 2013 .

[36]  Stacy B. Ehrlich,et al.  The importance of gesture in children's spatial reasoning. , 2006, Developmental psychology.

[37]  Hiroshi Ishii,et al.  Topobo: a constructive assembly system with kinetic memory , 2004, CHI.

[38]  Hiroshi Ishii,et al.  I/O brush: drawing with everyday objects as ink , 2004, CHI.

[39]  Kelly Dickerson,et al.  They can interact, but can they learn? Toddlers' transfer learning from touchscreens and television. , 2015, Journal of experimental child psychology.

[40]  Peta Wyeth,et al.  Electronic Blocks: Tangible Programming Elements for Preschoolers , 2001, INTERACT.

[41]  Juanita V. Copley,et al.  The Development of Spatial Skills Through Interventions Involving Block Building Activities , 2008 .

[42]  James A. Hendler,et al.  Designing PETS: a personal electronic teller of stories , 1999, CHI '99.

[43]  Seth E. Hunter,et al.  Make a Riddle and TeleStory: designing children's applications for the siftables platform , 2010, IDC.

[44]  Hiroshi Ishii,et al.  Tangible bits: beyond pixels , 2008, TEI.

[45]  Seymour Papert,et al.  Mindstorms: Children, Computers, and Powerful Ideas , 1981 .

[46]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[47]  Brian N. Verdine,et al.  Talking Shape: Parental Language with Electronic versus Traditional Shape Sorters. , 2015 .

[48]  Clinical Issues: Learning Prepositions , 2005 .

[49]  Allison Druin,et al.  The role of children in the design of new technology , 2002 .

[50]  Betty Sargeant,et al.  What is an ebook? What is a Book App? And Why Should We Care? An Analysis of Contemporary Digital Picture Books , 2015 .

[51]  Alissa Nicole Antle,et al.  Exploring how children use their hands to think: an embodied interactional analysis , 2013, Behav. Inf. Technol..

[52]  J. Sarama,et al.  Effects of a Preschool Mathematics Curriculum: Summative Research on the Building Blocks Project , 2007 .

[53]  J. Huttenlocher,et al.  Early sex differences in spatial skill. , 1999, Developmental psychology.

[54]  D. Clements Geometric and Spatial Thinking in Young Children. , 1998 .

[55]  Eva Cerezo,et al.  Bringing tabletop technology to all: evaluating a tangible farm game with kindergarten and special needs children , 2012, Personal and Ubiquitous Computing.

[56]  Hiroshi Ishii,et al.  mediaBlocks: physical containers, transports, and controls for online media , 1998, SIGGRAPH.

[57]  Roberta Michnick Golinkoff,et al.  Finding the missing piece: Blocks, puzzles, and shapes fuel school readiness , 2014, Trends in Neuroscience and Education.

[58]  Glen Bull Tutor, Tool, Tutee: A Vision Revisited , 2009 .

[59]  S. Goldin-Meadow,et al.  From action to abstraction: Gesture as a mechanism of change. , 2015, Developmental review : DR.

[60]  Koen van Boerdonk,et al.  Moving Tangible Interaction Systems to the Next Level , 2013, Computer.

[61]  Wilfried Brauer,et al.  Spatial Cognition III , 2003, Lecture Notes in Computer Science.

[62]  Janet C. Read,et al.  Child-computer interaction , 2013, Int. J. Child Comput. Interact..

[63]  J. Huttenlocher,et al.  Early puzzle play: a predictor of preschoolers' spatial transformation skill. , 2012, Developmental psychology.

[64]  Amy Bruckman,et al.  HCI FOR KIDS , 2012 .

[65]  Linda B. Smith From Fragments to Geometric Shape , 2009, Current directions in psychological science.

[66]  Alissa Nicole Antle,et al.  Knowledge gaps in hands-on tangible interaction research , 2012, ICMI '12.

[67]  Asim Evren Yantaç,et al.  Developing Transmedia Puzzle Play to Facilitate Spatial Skills of Preschoolers , 2016, IDC.

[68]  Alissa Nicole Antle,et al.  Are tangibles more fun?: comparing children's enjoyment and engagement using physical, graphical and tangible user interfaces , 2008, TEI.

[69]  D. Uttal,et al.  The malleability of spatial skills: a meta-analysis of training studies. , 2013, Psychological bulletin.

[70]  Oren Zuckerman,et al.  System blocks : learning about systems concepts through hands-on modeling and simulation , 2004 .

[71]  N. Newcombe,et al.  Taking shape: supporting preschoolers' acquisition of geometric knowledge through guided play. , 2013, Child development.

[72]  Hiroshi Ishii,et al.  Emerging frameworks for tangible user interfaces , 2000, IBM Syst. J..

[73]  Juan Pablo Hourcade,et al.  Child-Computer Interaction SIG: New Challenges and Opportunities , 2016, CHI Extended Abstracts.

[74]  David H. Uttal,et al.  Exploring and Enhancing Spatial Thinking , 2013 .

[75]  KERSTIN NACHTIGÄLLER,et al.  A story about a word: does narrative presentation promote learning of a spatial preposition in German two-year-olds? , 2013, Journal of child language.

[76]  Oren Zuckerman,et al.  To TUI or not to TUI: Evaluating performance and preference in tangible vs. graphical user interfaces , 2013, Int. J. Hum. Comput. Stud..

[77]  N. Newcombe,et al.  Block Talk: Spatial Language During Block Play , 2011 .

[78]  James A. Hendler,et al.  Designing StoryRooms: interactive storytelling spaces for children , 2000, DIS '00.

[79]  Janet C. Read,et al.  Bringing tabletop technologies to kindergarten children , 2009 .

[80]  Brian N. Verdine,et al.  Deconstructing building blocks: preschoolers' spatial assembly performance relates to early mathematical skills. , 2014, Child development.

[81]  Alyssa Friend Wise,et al.  Getting Down to Details: Using Theories of Cognition and Learning to Inform Tangible User Interface Design , 2013, Interact. Comput..

[82]  Allison Druin,et al.  KidPad: collaborative storytelling for children , 2002, CHI Extended Abstracts.

[83]  Paul Marshall,et al.  Editorial: the evolving field of tangible interaction for children: the challenge of empirical validation , 2012, Personal and Ubiquitous Computing.

[84]  Allison Druin,et al.  Methods and Techniques for Involving Children in the Design of New Technology for Children , 2013, Found. Trends Hum. Comput. Interact..

[85]  Allison Druin,et al.  Mixing ideas: a new technique for working with young children as design partners , 2004, IDC '04.

[86]  Ayah Bdeir,et al.  Getting Started with littleBits: Prototyping and Inventing with Modular Electronics , 2015 .

[87]  Janet C. Read,et al.  Evaluating Children's Interactive Products: Principles and Practices for Interaction Designers , 2008 .

[88]  Kathy Hirsh-Pasek,et al.  Guided Play: Where Curricular Goals Meet a Playful Pedagogy , 2013 .