Emerging frameworks for tangible user interfaces

We present steps toward a conceptual framework for tangible user interfaces. We introduce the MCRpd interaction model for tangible interfaces, which relates the role of physical and digital representations, physical control, and underlying digital models. This model serves as a foundation for identifying and discussing several key characteristics of tangible user interfaces. We identify a number of systems exhibiting these characteristics, and situate these within 12 application domains. Finally, we discuss tangible interfaces in the context of related research themes, both within and outside of the human-computer interaction domain.

[1]  Radia Perlman,et al.  Using Computer Technology to Provide a Creative Learning Environment for Preschool Children. AI Memo 360. , 1976 .

[2]  Roy Want,et al.  Bridging physical and virtual worlds with electronic tags , 1999, CHI '99.

[3]  M. Csíkszentmihályi,et al.  The meaning of things: Coding categories and definitions , 1981 .

[4]  Darren Leigh,et al.  Building virtual structures with physical blocks , 1999, UIST '99.

[5]  Stephen L. Burbeck,et al.  Applications programming in smalltalk-80: how to use model-view-controller (mvc) , 1987 .

[6]  Hiroshi Ishii,et al.  The metaDESK: models and prototypes for tangible user interfaces , 1997, UIST '97.

[7]  John Seely Brown,et al.  The coming age of calm technolgy , 1997 .

[8]  R McGee David,et al.  Use what you''ve got: Steps toward opportunistic computing , 2000 .

[9]  Wendy E. Mackay,et al.  Augmented reality: linking real and virtual worlds: a new paradigm for interacting with computers , 1998, AVI '98.

[10]  Jakob Nielsen,et al.  The Anti-Mac interface , 1996, CACM.

[11]  Karon E. MacLean,et al.  Tagged handles: merging discrete and continuous manual control , 2000, CHI.

[12]  Margaret M. Burnett,et al.  Visual Programming , 1995 .

[13]  Y. Guiard.  Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model. , 1987, Journal of motor behavior.

[14]  Lionel Médini,et al.  Reinventing the familiar: exploring an augmented reality design space for air traffic control , 1998, CHI.

[15]  Gerhard Fischer,et al.  Enhancing communication, facilitating shared understanding, and creating better artifacts by integrating physical and computational media for design , 1997, DIS '97.

[16]  Hiroshi Ishii,et al.  TouchCounters: designing interactive electronic labels for physical containers , 1999, CHI '99.

[17]  Ken Hinckley,et al.  Passive real-world interface props for neurosurgical visualization , 1994, CHI '94.

[18]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[19]  Nan C. Shu,et al.  Visual Programming: Perspectives and Approaches , 1989, IBM Syst. J..

[20]  Lars Erik Holmquist,et al.  Token-Based Acces to Digital Information , 1999, HUC.

[21]  Friedrich Wilhelm Bruns,et al.  A new approach to human-computer interaction—synchronous modelling in real and virtual spaces , 1997, DIS '97.

[22]  Lisa Stifelman,et al.  Augmenting real-world objects: a paper-based audio notebook , 1996, CHI Conference Companion.

[23]  Kimiyoshi Machii,et al.  Retrieving electronic documents with real-world objects on InteractiveDESK , 1995, UIST '95.

[24]  Hiroshi Ishii,et al.  curlybot: designing a new class of computational toys , 2000, CHI.

[25]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[26]  Donald A. Norman,et al.  Things that make us smart , 1979 .

[27]  Craig Alexander Wisneski,et al.  The Design of Personal Ambient Displays , 1999 .

[28]  Marina Umaschi.  Soft toys with computer hearts: building personal storytelling environments , 1997, CHI Extended Abstracts.

[29]  Robert Aish,et al.  Architecture without numbers — CAAD based on a 3D modelling system , 1984 .

[30]  Hiroshi Ishii,et al.  Illuminating light: an optical design tool with a luminous-tangible interface , 1998, CHI.

[31]  Norbert A. Streitz,et al.  i-LAND: an interactive landscape for creativity and innovation , 1999, CHI '99.

[32]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..

[33]  Lia Adams,et al.  Palette: a paper interface for giving presentations , 1999, CHI '99.

[34]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[35]  Brygg Ullmer,et al.  Models and mechanisms for tangible user interfaces , 1997 .

[36]  Kwindla Hultman Kramer,et al.  Moveable Objects, Mobile Code , 1998 .

[37]  Marian Petre,et al.  Why looking isn't always seeing: readership skills and graphical programming , 1995, CACM.

[38]  John J. Leggett,et al.  Seven Issues, Revisited , 2002, HYPERTEXT '02.

[39]  Hiroshi Ishii,et al.  Bricks: laying the foundations for graspable user interfaces , 1995, CHI '95.

[40]  Tomas Sokoler,et al.  AROMA: abstract representation of presence supporting mutual awareness , 1997, CHI.

[41]  G. Halasz Frank,et al.  Reflections on NoteCards: seven issues for the next generation of hypermedia systems , 1988 .

[42]  William J. Mitchell,et al.  Rethinking the book , 1999 .

[43]  Mark C. Detweiler,et al.  Iconic Reference: Evolving Perspectives and an Organizing Framework , 1993, Int. J. Man Mach. Stud..

[44]  Jeff A. Johnson,et al.  The Xerox Star: a retrospective , 1989, Computer.

[45]  Philip R. Cohen,et al.  Creating tangible interfaces by augmenting physical objects with multimodal language , 2001, IUI '01.

[46]  William Buxton,et al.  Graspable user interfaces , 1996 .

[47]  Frank G. Halasz,et al.  Reflections on NoteCards: seven issues for the next generation of hypermedia systems , 1987, Hypertext.

[48]  Itiro Siio.  InfoBinder: A pointing device for a virtual desktop system , 1995, HCI International 1995.

[49]  Sean White,et al.  Tangible progress: less is more in Somewire audio spaces , 1999, CHI '99.

[50]  Jonathan Cohen,et al.  Logjam: a tangible multi-person interface for video logging , 1999, CHI '99.

[51]  Joseph A. Paradiso,et al.  PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play , 1999, CHI '99.

[52]  G. Brahm,et al.  Prosthetic Territories: Politics And Hypertechnologies , 1995 .

[53]  Hiroshi Ishii,et al.  mediaBlocks: physical containers, transports, and controls for online media , 1998, SIGGRAPH.

[54]  Hiroshi Ishii,et al.  Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information , 1998, CoBuild.

[55]  Yvonne Rogers,et al.  External cognition: how do graphical representations work? , 1996, Int. J. Hum. Comput. Stud..

[56]  Joëlle Coutaz,et al.  PAC, an Object Oriented Model for Dialog Design , 1987 .

[57]  K Sloan,et al.  The hand that rocks the cradle. , 1987, Film History.

[58]  Richard Daniel Borovoy.  Genuine object oriented programming , 1996 .

[59]  Hideaki Kuzuoka,et al.  Mediating awareness and communication through digital but physical surrogates , 1999, CHI Extended Abstracts.

[60]  Stephanie Houde,et al.  Working towards rich and flexible file representations , 1993, INTERCHI Adjunct Proceedings.

[61]  A. D. Manning,et al.  Understanding Comics: The Invisible Art , 1993 .

[62]  Mitchel Resnick,et al.  Digital manipulatives: new toys to think with , 1998, CHI.

[63]  Jiajie Zhang,et al.  Representations in Distributed Cognitive Tasks , 1994, Cogn. Sci..

[64]  Tim McNerney,et al.  Tangible programming bricks : an approach to making programming accessible to everyone , 1999 .

[65]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[66]  Thomas P. Moran,et al.  Embodied User Interfaces: Towards Invisible User Interfaces , 1998, EHCI.

[67]  Dennis Proffitt,et al.  Two-handed virtual manipulation , 1998, TCHI.

[68]  G. W. Teuscher.  The hand that rocks the cradle... , 1988, ASDC journal of dentistry for children.

[69]  William Buxton,et al.  Evolution of a reactive environment , 1995, CHI '95.

[70]  Hiroshi Ishii,et al.  mediaBlocks: tangible interfaces for online media , 1999, CHI Extended Abstracts.

[71]  Mitchel Resnick,et al.  Behavior construction kits , 1993, CACM.

[72]  Hiroshi Ishii,et al.  Water lamp and pinwheels: ambient projection of digital information into architectural space , 1998, CHI Conference Summary.

[73]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.

[74]  Scott E. Hudson,et al.  The information percolator: ambient information display in a decorative object , 1999, UIST '99.

[75]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[76]  W. Gropp,et al.  Does increased aromatase activity in adipose fibroblasts cause low sexual desire in patients with HIV lipodystrophy? , 2002, Sexually transmitted infections.

[77]  Hiroshi Ishii,et al.  Tangible interfaces for remote collaboration and communication , 1998, CSCW '98.

[78]  Robert Aish.  3D input for CAAD systems , 1979 .

[79]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[80]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[81]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[82]  David Frohlich,et al.  Direct manipulation and other lessons , 1997 .

[83]  J. Michael McCarthy,et al.  Introduction to theoretical kinematics , 1990 .

[84]  Justine Cassell,et al.  Rosebud: technological toys for storytelling , 1997, CHI Extended Abstracts.

[85]  Hiroshi Ishii,et al.  Triangles: tangible interface for manipulation and exploration of digital information topography , 1998, CHI.

[86]  Hiroshi Ishii,et al.  Emancipated pixels: real-world graphics in the luminous room , 1999, SIGGRAPH.

[87]  John H. Frazer,et al.  An Evolutionary Architecture , 1995 .

[88]  Morten Fjeld,et al.  BUILD-IT: An Intuitive Design Tool Based on Direct Object Manipulation , 1997, Gesture Workshop.

[89]  Matthew G. Gorbet,et al.  Beyond input devices : a new conceptual framework for the design of physical-digital objects , 1998 .

[90]  Pierre David Wellner,et al.  Interacting with paper on the DigitalDesk , 1993, CACM.

[91]  Wendy E. Mackay,et al.  Video mosaic: laying out time in a physical space , 1994, MULTIMEDIA '94.

[92]  Paul A. Beardsley,et al.  Tangible interaction + graphical interpretation: a new approach to 3D modeling , 2000, SIGGRAPH.

[93]  Jiajie Zhang,et al.  The Nature of External Representations in Problem Solving , 1997, Cogn. Sci..

[94]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..