shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction

We explore interactions enabled by 2D spatial manipulation and self-actuation of a tabletop shape display. To explore these interactions, we developed shapeShift, a compact, high-resolution (7 mm pitch), mobile tabletop shape display. shapeShift can be mounted on passive rollers allowing for bimanual interaction where the user can freely manipulate the system while it renders spatially relevant content. shapeShift can also be mounted on an omnidirectional-robot to provide both vertical and lateral kinesthetic feedback, display moving objects, or act as an encountered-type haptic device for VR. We present a study on haptic search tasks comparing spatial manipulation of a shape display for egocentric exploration of a map versus exploration using a fixed display and a touch pad. Results show a 30% decrease in navigation path lengths, 24% decrease in task time, 15% decrease in mental demand and 29% decrease in frustration in favor of egocentric navigation.

[1]  Takeo Kanade,et al.  What you can see is what you can feel-development of a visual/haptic interface to virtual environment , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[2]  Marcel Worring,et al.  Navigating on hand held displays: Dynamic versus Static Keyhole Navigation , 2006 .

[3]  Jan O. Borchers,et al.  Madgets: actuating widgets on interactive tabletops , 2010, UIST.

[4]  Christopher R. Wagner,et al.  A tactile shape display using RC servomotors , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[5]  Helge J. Ritter,et al.  An integrated multi-modal actuated tangible user interface for distributed collaborative planning , 2012, Tangible and Embedded Interaction.

[6]  Victor Kaptelinin,et al.  Item recognition in menu selection: the effect of practice , 1993, INTERCHI Adjunct Proceedings.

[7]  Mark S. Sanders,et al.  Human Factors in Engineering and Design , 1957 .

[8]  Michitaka Hirose,et al.  Simulation and presentation of curved surface in virtual reality environment through surface display , 1995, Proceedings Virtual Reality Annual International Symposium '95.

[9]  Johannes Schöning,et al.  Map navigation with mobile devices: virtual versus physical movement with and without visual context , 2007, ICMI '07.

[10]  S J Lederman,et al.  Exploring environments by hand or foot: time-based heuristics for encoding distance in movement space. , 1987, Journal of experimental psychology. Learning, memory, and cognition.

[11]  Jonathan Grudin,et al.  Human Computer Interaction: The Year 2000 and Beyond , 1995, HCI.

[12]  Pierre Dragicevic,et al.  Zooids: Building Blocks for Swarm User Interfaces , 2016, UIST.

[13]  Pattie Maes,et al.  Flexpad: highly flexible bending interactions for projected handheld displays , 2013, CHI.

[14]  George W. Fitzmaurice,et al.  Situated information spaces and spatially aware palmtop computers , 1993, CACM.

[15]  Antonio Krüger,et al.  Investigating the effectiveness of peephole interaction for smartwatches in a map navigation task , 2014, MobileHCI '14.

[16]  Cesare Cornoldi,et al.  Remembering Object Position in the Absence of Vision: Egocentric, Allocentric, and Egocentric Decentred Frames of Reference , 2007, Perception.

[17]  Marc O. Ernst,et al.  Visually Guided Haptic Search , 2010, IEEE Transactions on Haptics.

[18]  Hiroo Iwata,et al.  Project FEELEX: adding haptic surface to graphics , 2001, SIGGRAPH.

[19]  J W Garrett,et al.  The Adult Human Hand: Some Anthropometric and Biomechanical Considerations , 1971, Human factors.

[20]  Masahiko Inami,et al.  Remote active tangible interactions , 2007, TEI.

[21]  James N. MacGregor,et al.  Minimizing User Search Time in Menu Retrieval Systems , 1985 .

[22]  Roman Rädle,et al.  The effect of egocentric body movements on users' navigation performance and spatial memory in zoomable user interfaces , 2013, ITS.

[23]  Daniel Leithinger,et al.  Grasping information and collaborating through shape displays , 2015 .

[24]  Jacqueline C. Snow,et al.  Real-world objects are more memorable than photographs of objects , 2014, Front. Hum. Neurosci..

[25]  Jun Rekimoto,et al.  Lumen: interactive visual and shape display for calm computing , 2004, SIGGRAPH '04.

[26]  R. Bjork,et al.  Recency-sensitive retrieval processes in long-term free recall☆ , 1974 .

[27]  J R Lackner,et al.  Proprioceptive influences on auditory and visual spatial localization , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[28]  E. Brenner,et al.  The use of proprioception and tactile information in haptic search. , 2008, Acta psychologica.

[29]  Brian D. Ehret,et al.  Learning where to look: location learning in graphical user interfaces , 2002, CHI.

[30]  Hiroshi Ishii,et al.  Relief: a scalable actuated shape display , 2010, TEI '10.

[31]  Susan J. Lederman,et al.  How well can we encode spatial layout from sparse kinesthetic contact? , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[32]  William A. McNeely,et al.  Robotic graphics: a new approach to force feedback for virtual reality , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[33]  Majken Kirkegaard Rasmussen,et al.  Shape-changing interfaces: a review of the design space and open research questions , 2012, CHI.

[34]  Hiroshi Ishii,et al.  Recompose: direct and gestural interaction with an actuated surface , 2011, CHI EA '11.

[35]  Hiroshi Ishii,et al.  The metaDESK: models and prototypes for tangible user interfaces , 1997, UIST '97.

[36]  D. Maynes-Aminzade,et al.  The actuated workbench: computer-controlled actuation in tabletop tangible interfaces , 2003, ACM Trans. Graph..

[37]  Humberto Campanella,et al.  Tactile Acuity Charts: A Reliable Measure of Spatial Acuity , 2014, PloS one.

[38]  Daniel J. Wigdor,et al.  Snake Charmer: Physically Enabling Virtual Objects , 2016, TEI.

[39]  Scott R. Klemmer,et al.  Two worlds apart: bridging the gap between physical and virtual media for distributed design collaboration , 2003, CHI '03.

[40]  Hiroshi Ishii,et al.  inFORM: dynamic physical affordances and constraints through shape and object actuation , 2013, UIST.

[41]  Lawrence H. Kim,et al.  Robotic Assembly of Haptic Proxy Objects for Tangible Interaction and Virtual Reality , 2017, ISS.

[42]  Dennis Proffitt,et al.  Attention and visual feedback: the bimanual frame of reference , 1997, SI3D.

[43]  Gary W. Meyer,et al.  A handheld flexible display system , 2005, VIS 05. IEEE Visualization, 2005..

[44]  Makoto Shimojo,et al.  Human shape recognition performance for 3D tactile display , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[45]  Hiroshi Ishii,et al.  Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration , 2014, UIST.

[46]  Sebastian Boring,et al.  Hover Pad: interacting with autonomous and self-actuated displays in space , 2014, UIST.

[47]  Shumin Zhai,et al.  Performance evaluation of input devices in trajectory-based tasks: an application of the steering law , 1999, CHI '99.

[48]  Mike Sinclair,et al.  TouchMover: actuated 3D touchscreen with haptic feedback , 2013, ITS.

[49]  Roman Rädle,et al.  Bigger is not always better: display size, performance, and task load during peephole map navigation , 2014, CHI.

[50]  Christine M. Haslegrave,et al.  Bodyspace: Anthropometry, Ergonomics And The Design Of Work , 1986 .

[51]  Dennis Proffitt,et al.  Cooperative bimanual action , 1997, CHI.