Characterizing Tangible Interaction During a Creative Combination Task

Tangible user interfaces change the way we interact with digital information, with physical affordances that are distinctly different from pointing and keyboard/mouse interaction. As a precursor to studying the impact of tangible interfaces on design cognition, this paper presents a coding scheme for measuring the difference between two types of user interfaces: tangible and pointing. We perform a case study, using data collected from an experiment in which participants are asked to make word combinations from a set of six nouns and give them meaning. The task is presented as a design task with references to function, behavior, and structure of the word combination meanings. The case study shows large differences in gesture and action between the two conditions. We conclude with hypotheses on how interaction modalities that afford more body movement may have an impact on creativity and design cognition.

[1]  S. Goldin-Meadow,et al.  Gesturing makes learning last , 2008, Cognition.

[2]  Pieter Pauwels,et al.  Conversation and Critique within the Architectural Design Process: A Linkograph Analysis , 2015 .

[3]  Katherine A. Brady,et al.  Modeling Expectation for Evaluating Surprise in Design Creativity , 2015 .

[4]  Margot Brereton,et al.  An observational study of how objects support engineering design thinking and communication: implications for the design of tangible media , 2000, CHI.

[5]  Susan Goldin-Meadow,et al.  Gesturing has a larger impact on problem-solving than action, even when action is accompanied by words , 2015, Language, cognition and neuroscience.

[6]  D. Hofstadter.  Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought, Douglas Hofstadter. 1994. Basic Books, New York, NY. 512 pages. ISBN: 0-465-05154-5. $30.00 , 1995 .

[7]  Susan Goldin-Meadow,et al.  Action’s Influence on Thought: The Case of Gesture , 2010, Perspectives on psychological science : a journal of the Association for Psychological Science.

[8]  Allen Bevans.  Investigating the effects of bimanual multitouch interaction on creativity , 2011, C&C '11.

[9]  R. Sternberg,et al.  The concept of creativity: Prospects and paradigms. , 1998 .

[10]  D. Gentner,et al.  Chapter 10 On the Combinatorial Semantics of Noun Pairs: Minor and Major Adjustments to Meaning , 1991 .

[11]  J. P. Foley,et al.  Gesture and Environment , 1942 .

[12]  Judy Kay,et al.  Gesture-based interaction design: communication and cognition , 2014, CHI Extended Abstracts.

[13]  Paul P. Maglio,et al.  On Distinguishing Epistemic from Pragmatic Action , 1994, Cogn. Sci..

[14]  Andy Dong,et al.  The Use and Misuse of the Concept of Affordance , 2015 .

[15]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[16]  Morten Fjeld,et al.  BUILD-IT: An Intuitive Design Tool Based on Direct Object Manipulation , 1997, Gesture Workshop.

[17]  John S. Gero,et al.  Design Prototypes: A Knowledge Representation Schema for Design , 1990, AI Mag..

[18]  Mary Lou Maher,et al.  The Impact of Tangible User Interfaces on Designers' Spatial Cognition , 2008, Hum. Comput. Interact..

[19]  Max Weisbuch,et al.  Fluid Movement and Fluid Social Cognition , 2014, Personality & social psychology bulletin.

[20]  Petra Badke-Schaub,et al.  Mental Models and Creativity in Engineering and Architectural Design Teams , 2015 .

[21]  Barbara Tversky,et al.  Using Diagrams and Gestures to Think and Talk about Insight Problems , 2006 .

[22]  Mary Lou Maher,et al.  Evaluating creativity in humans, computers, and collectively intelligent systems , 2010, DESIRE.

[23]  Alissa Nicole Antle,et al.  Are tangibles more fun?: comparing children's enjoyment and engagement using physical, graphical and tangible user interfaces , 2008, TEI.

[24]  Hiroshi Ishii,et al.  Bricks: laying the foundations for graspable user interfaces , 1995, CHI '95.

[25]  Mary Lou Maher,et al.  Conceptual Combination Modulated by Action using Tangible Computers , 2015, CogSci.