Project Zanzibar: A Portable and Flexible Tangible Interaction Platform

We present Project Zanzibar: a flexible mat that can locate, uniquely identify and communicate with tangible objects placed on its surface, as well as sense a user's touch and hover hand gestures. We describe the underlying technical contributions: efficient and localised Near Field Communication (NFC) over a large surface area; object tracking combining NFC signal strength and capacitive footprint detection, and manufacturing techniques for a rollable device form-factor that enables portability, while providing a sizable interaction area when unrolled. In addition, we detail design patterns for tangibles of varying complexity and interactive capabilities, including the ability to sense orientation on the mat, harvest power, provide additional input and output, stack, or extend sensing outside the bounds of the mat. Capabilities and interaction modalities are illustrated with self-generated applications. Finally, we report on the experience of professional game developers building novel physical/digital experiences using the platform.

[1]  Pierre Wellner.  The DigitalDesk calculator: tangible manipulation on a desk top display , 1991, UIST '91.

[2]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[3]  Paul Dourish,et al.  Where the action is , 2001 .

[4]  Christian Holz,et al.  Biometric Touch Sensing: Seamlessly Augmenting Each Touch with Continuous Authentication , 2015, UIST.

[5]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[6]  Alanson P. Sample,et al.  IDSense: A Human Object Interaction Detection System Based on Passive UHF RFID , 2015, CHI.

[7]  Hiroshi Ishii,et al.  Sensetable: a wireless object tracking platform for tangible user interfaces , 2001, CHI.

[8]  Hiroshi Ishii,et al.  Illuminating clay: a 3-D tangible interface for landscape analysis , 2002, CHI.

[9]  Andrew D. Wilson.  PlayAnywhere: a compact interactive tabletop projection-vision system , 2005, UIST.

[10]  William Buxton,et al.  Graspable user interfaces , 1996 .

[11]  Guillaume Levieux,et al.  Designing tangible video games: lessons learned from the sifteo cubes , 2014, CHI.

[12]  Kenton O'Hara,et al.  Pre-Touch Sensing for Mobile Interaction , 2016, CHI.

[13]  Arnan Sipitakiat,et al.  Robo-Blocks: designing debugging abilities in a tangible programming system for early primary school children , 2012, IDC '12.

[14]  Sergi Jordà,et al.  The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces , 2007, TEI.

[15]  Alex Olwal,et al.  SurfaceFusion: unobtrusive tracking of everyday objects in tangible user interfaces , 2008, Graphics Interface.

[16]  Peter Gerjets,et al.  LEGO music: learning composition with bricks , 2015, IDC.

[17]  Michael S. Horn,et al.  Tangible programming and informal science learning: making TUIs work for museums , 2008, IDC.

[18]  Alanson P. Sample,et al.  RapID: A Framework for Fabricating Low-Latency Interactive Objects with RFID Tags , 2016, CHI.

[19]  Hiroshi Ishii,et al.  Bricks: laying the foundations for graspable user interfaces , 1995, CHI '95.

[20]  Jan O. Borchers,et al.  PERCs: Persistently Trackable Tangibles on Capacitive Multi-Touch Displays , 2015, UIST.

[21]  Patrick Baudisch,et al.  Fiberio: a touchscreen that senses fingerprints , 2013, UIST.

[22]  Alissa Nicole Antle,et al.  The CTI framework: informing the design of tangible systems for children , 2007, TEI.

[23]  David Wetherall,et al.  RFID sensor networks with the Intel WISP , 2008, SenSys '08.

[24]  Hideyuki Suzuki,et al.  Interaction-level support for collaborative learning: AlgoBlock - an open programming language , 1995, CSCL.

[25]  J. Underkoff.  Urp : A Luminous-Tangible Workbench for Urban Planning and Design , 1999, CHI 1999.

[26]  Jun Rekimoto,et al.  DataTiles: a modular platform for mixed physical and graphical interactions , 2001, CHI.

[27]  Jun Rekimoto,et al.  SmartSkin: an infrastructure for freehand manipulation on interactive surfaces , 2002, CHI.

[28]  Orit Shaer,et al.  Smart Blocks: a tangible mathematical manipulative , 2007, TEI.

[29]  Mark D. Gross,et al.  roBlocks: a robotic construction kit for mathematics and science education , 2006, ICMI '06.

[30]  Hiroshi Ishii,et al.  The tangible user interface and its evolution , 2008, CACM.

[31]  Pattie Maes,et al.  Siftables: towards sensor network user interfaces , 2007, TEI.

[32]  Paul H. Dietz,et al.  SurfaceWare: dynamic tagging for Microsoft Surface , 2009, Tangible and Embedded Interaction.

[33]  Seymour Papert,et al.  Mindstorms: Children, Computers, and Powerful Ideas , 1981 .

[34]  James D. Hollan,et al.  SLAP widgets: bridging the gap between virtual and physical controls on tabletops , 2009, CHI.

[35]  Mitchel Resnick,et al.  Extending tangible interfaces for education: digital montessori-inspired manipulatives , 2005, CHI.

[36]  Ian Oakley,et al.  Designing tangible magnetic appcessories , 2013, TEI '13.

[37]  Scott Brenner Brave.  Tangible interfaces for remote communication and collaboration , 1998 .

[38]  Hendrik Müller,et al.  Understanding tablet use: a multi-method exploration , 2012, Mobile HCI.

[39]  Paula Te.  TADCAD: a tangible and gestural 3D modeling & printing platform for building creativity , 2015, IDC.

[40]  Hongan Wang,et al.  T-Maze: a tangible programming tool for children , 2011, IDC.

[41]  Matthew S. Reynolds,et al.  Finding Common Ground: A Survey of Capacitive Sensing in Human-Computer Interaction , 2017, CHI.

[42]  Senaka Buthpitiya,et al.  Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts , 2015, CHI.

[43]  Roy Want,et al.  Bridging physical and virtual worlds with electronic tags , 1999, CHI '99.

[44]  Yvonne Rogers,et al.  Using 'tangibles' to promote novel forms of playful learning , 2003, Interact. Comput..

[45]  Li-Wei Chan,et al.  TUIC: enabling tangible interaction on capacitive multi-touch displays , 2011, CHI.