Towards a framework for investigating tangible environments for learning

External representations have been shown to play a key role in mediating cognition. Tangible environments offer the opportunity for novel representational formats and combinations, potentially increasing representational power for supporting learning. However, we currently know little about the specific learning benefits of tangible environments, and have no established framework within which to analyse the ways that external representations work in tangible environments to support learning. Taking external representation as the central focus, this paper proposes a framework for investigating the effect of tangible technologies on interaction and cognition. Key artefact-action-representation relationships are identified, and classified to form a structure for investigating the differential cognitive effects of these features. An example scenario from our current research is presented to illustrate how the framework can be used as a method for investigating the effectiveness of differential designs for supporting science learning.

[1]  K. H. Stauder,et al.  Psychology of the Child , 1959 .

[2]  E. M. Adams,et al.  On Knowing that , 1958 .

[3]  Seymour Papert,et al.  Mindstorms: Children, Computers, and Powerful Ideas , 1981 .

[4]  G. Lakoff,et al.  Metaphors We Live by , 1982 .

[5]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..

[6]  P. Johnson-Laird,et al.  PSYCHOLOGICAL SCIENCE Research Article HOW DIAGRAMS CAN IMPROVE REASONING , 2022 .

[7]  Jiajie Zhang,et al.  Representations in Distributed Cognitive Tasks , 1994, Cogn. Sci..

[8]  Yvonne Rogers,et al.  External cognition: how do graphical representations work? , 1996, Int. J. Hum. Comput. Stud..

[9]  Mary Target,et al.  Society for Research in Child Development , 1997 .

[10]  Richard Cox,et al.  Intelligence and Multimodality in Multimedia Interfaces: Research and Applications , 1997, AAAI 1997.

[11]  Yvonne Rogers,et al.  How can interactive multimedia facilitate learning , 1998 .

[12]  Mitchel Resnick,et al.  Technologies for lifelong kindergarten , 1998 .

[13]  Mitchel Resnick,et al.  Digital manipulatives: new toys to think with , 1998, CHI.

[14]  H. Ishii,et al.  Urp: a luminous-tangible workbench for urban planning and design , 1999, CHI '99.

[15]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[16]  Lori L. Scarlatos,et al.  TICLE: a tangible interface for collaborative learning environments , 1999, CHI Extended Abstracts.

[17]  D. Clements.  ‘Concrete’ Manipulatives, Concrete Ideas , 2000 .

[18]  Sara Jones,et al.  Animated Diagrams: An Investigation into the Cognitive Effects of Using Animation to Illustrate Dynamic Processes , 2000, Diagrams.

[19]  Hiroshi Ishii,et al.  Emerging frameworks for tangible user interfaces , 2000, IBM Syst. J..

[20]  Edward Childs.  THE SONIFICATION OF NUMERICAL FLUID FLOW SIMULATIONS , 2001 .

[21]  Yvonne Rogers,et al.  How many ways can you mix colour? Young children's explorations of mixed reality environments , 2001 .

[22]  Paul Dourish,et al.  Where the action is , 2001 .

[23]  Hiroshi Ishii,et al.  Sensetable: a wireless object tracking platform for tangible user interfaces , 2001, CHI.

[24]  M. Garcia-Ruiz.  USING NON-SPEECH SOUNDS TO CONVEY MOLECULAR PROPERTIES IN A VIRTUAL ENVIRONMENT , 2001 .

[25]  Yvonne Rogers,et al.  Things aren't what they seem to be: innovation through technology inspiration , 2002, DIS '02.

[26]  Sara Price,et al.  Animated Diagrams: How effective are explicit dynamics for learners? , 2002 .

[27]  Stephen Brewster,et al.  Nonspeech auditory output , 2002 .

[28]  Wyeth Peta,et al.  Tangible programming elements for young children , 2002, CHI Extended Abstracts.

[29]  Steve Benford,et al.  A Framework for Tangible User Interfaces , 2003 .

[30]  Yvonne Rogers,et al.  Using 'tangibles' to promote novel forms of playful learning , 2003, Interact. Comput..

[31]  Yvonne Rogers,et al.  Conceptualising tangibles to support learning , 2003, IDC '03.

[32]  Gerd Kortuem,et al.  Exploring cube affordance: towards a classification of non-verbal dynamics of physical interfaces for wearable computing , 2003 .

[33]  Sergi Jordà,et al.  Sonigraphical Instruments: From FMOL to the reacTable* , 2003, NIME.

[34]  R. Pea,et al.  Seeing What We Build Together: Distributed Multimedia Learning Environments for Transformative Communications , 2004 .

[35]  Kenneth P. Fishkin,et al.  A taxonomy for and analysis of tangible interfaces , 2004, Personal and Ubiquitous Computing.

[36]  Danae Stanton Fraser,et al.  Literature Review in Learning with Tangible Technologies , 2004 .

[37]  Yvonne Rogers,et al.  Let's get physical: The learning benefits of interacting in digitally augmented physical spaces , 2004, Comput. Educ..

[38]  Lorna M. Brown,et al.  Tactons: Structured Tactile Messages for Non-Visual Information Display , 2004, AUIC.

[39]  Yvonne Rogers,et al.  The Ambient Horn: designing a novel audio-based learning experience , 2004, Personal and Ubiquitous Computing.

[40]  Hiroshi Ishii,et al.  Topobo: a constructive assembly system with kinetic memory , 2004, CHI.

[41]  Martin R. Gibbs,et al.  Mediating intimacy: designing technologies to support strong-tie relationships , 2005, CHI.

[42]  Mitchel Resnick,et al.  Extending tangible interfaces for education: digital montessori-inspired manipulatives , 2005, CHI.

[43]  湯淺 太一,et al.  20世紀の名著名論:Seymour Papert: Mindstorms:Children Computers and Powerful Ideas Basic Books New York 1980 , 2005 .

[44]  Tom Moher,et al.  Roomquake: embedding dynamic phenomena within the physical space of an elementary school classroom , 2005, CHI EA '05.

[45]  Robert L. Goldstone,et al.  The Transfer of Scientific Principles Using Concrete and Idealized Simulations , 2005, Journal of the Learning Sciences.

[46]  Jacob Buur,et al.  Getting a grip on tangible interaction: a framework on physical space and social interaction , 2006, CHI.

[47]  V. Prain,et al.  An Exploratory Study of Teachers’ and Students’ Use of Multi‐modal Representations of Concepts in Primary Science , 2006 .

[48]  Hiroshi Ishii,et al.  Beyond record and play: backpacks: tangible modulators for kinetic behavior , 2006, CHI.

[49]  Alan F. Blackwell,et al.  Correlates of the cognitive dimensions for tangible user interface , 2006, J. Vis. Lang. Comput..

[50]  Rin-ichiro Taniguchi,et al.  A Tangible Interface for Hands-on Learning , 2006, Tenth International Conference on Information Visualisation (IV'06).

[51]  Amanda J. Parkes,et al.  Beyond record and play , 2006 .

[52]  Oren Zuckerman,et al.  Flow blocks as a conceptual bridge between understanding the structure and behavior of a complex causal system , 2006 .

[53]  Christian Heath,et al.  Proceedings of CHI 2006 , 2006 .

[54]  Paul Marshall,et al.  Do tangible interfaces enhance learning? , 2007, TEI.

[55]  Stephen A. Brewster,et al.  Mobile Multi-actuator Tactile Displays , 2007, HAID.

[56]  Alissa Nicole Antle,et al.  The CTI framework: informing the design of tangible systems for children , 2007, TEI.

[57]  Alissa Nicole Antle,et al.  Designing tangibles for children: what designers need to know , 2007, CHI Extended Abstracts.

[58]  Jiao Jian-li.  Literature Review in Learning with Tangible Technologies , 2008 .

[59]  Mark D. Gross,et al.  The robot is the program: interacting with roBlocks , 2008, Tangible and Embedded Interaction.

[60]  三嶋 博之.  The theory of affordances , 2008 .

[61]  Alissa Nicole Antle,et al.  Are tangibles more fun?: comparing children's enjoyment and engagement using physical, graphical and tangible user interfaces , 2008, TEI.

[62]  Sara Price,et al.  A representation approach to conceptualizing tangible learning environments , 2008, TEI.

[63]  H. Gellersen,et al.  TEI'08 : second international conference on tangible and embedded interaction : conference proceedings, February 18-20, 2008, Bonn, Germany , 2008 .

[64]  Chapter 12. Non-speech Auditory Output , 2022 .