Mediate: A Spatial Tangible Interface for Mixed Reality

Recent Virtual Reality (VR) systems render highly immersive visual experiences, yet currently lack tactile feedback for feeling virtual objects with our hands and bodies. Shape Displays offer solid tangible interaction but have not been integrated with VR or have been restricted to desktop-scale workspaces. This work represents a fusion of mobile robotics, haptic props, and shape-display technology and commercial Virtual Reality to overcome these limitations. We present Mediate, a semi-autonomous mobile shape-display that locally renders 3D physical geometry co-located with room-sized virtual environments as a conceptual step towards large-scale tangible interaction in Virtual Reality. We compare this "dynamic just-in-time mockup" concept to other haptic paradigms and discuss future applications and interaction scenarios.

[1]  Hiroshi Ishii,et al.  inFORM: dynamic physical affordances and constraints through shape and object actuation , 2013, UIST.

[2]  Lawrence H. Kim,et al.  Robotic Assembly of Haptic Proxy Objects for Tangible Interaction and Virtual Reality , 2017, ISS.

[3]  Takeo Kanade,et al.  WYSIWYF Display: A Visual/Haptic Interface to Virtual Environment , 1999, Presence.

[4]  Michitaka Hirose,et al.  Simulation and presentation of curved surface in virtual reality environment through surface display , 1995, Proceedings Virtual Reality Annual International Symposium '95.

[5]  William A. McNeely,et al.  Robotic graphics: a new approach to force feedback for virtual reality , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[6]  Hiroshi Ishii,et al.  Sublimate: state-changing virtual and physical rendering to augment interaction with shape displays , 2013, CHI.

[7]  Markus Löchtefeld,et al.  Morphees: toward high "shape resolution" in self-actuated flexible mobile devices , 2013, CHI.

[8]  I. V. Ramakrishnan,et al.  Tactile Accessibility: Does Anyone Need a Haptic Glove? , 2016, ASSETS.

[9]  Sean Follmer,et al.  shapeShift: A Mobile Tabletop Shape Display for Tangible and Haptic Interaction , 2017, UIST.

[10]  Daniel J. Wigdor,et al.  Snake Charmer: Physically Enabling Virtual Objects , 2016, TEI.

[11]  Sriram Subramanian,et al.  UltraHaptics: multi-point mid-air haptic feedback for touch surfaces , 2013, UIST.

[12]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[13]  Zhenyi He,et al.  Robotic Haptic Proxies for Collaborative Virtual Reality , 2017, ArXiv.

[14]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[15]  Hiroshi Ishii,et al.  Haptic Edge Display for Mobile Tactile Interaction , 2016, CHI.

[16]  Hiroshi Ishii,et al.  Shape Displays: Spatial Interaction with Dynamic Physical Form , 2015, IEEE Computer Graphics and Applications.

[17]  Hiroo Iwata,et al.  Project FEELEX: adding haptic surface to graphics , 2001, SIGGRAPH.