RetroShape: Leveraging Rear-Surface Shape Displays for 2.5D Interaction on Smartwatches

The small screen size of a smartwatch limits user experience when watching or interacting with media. We propose a supplementary tactile feedback system to enhance the user experience with a method unique to the smartwatch form factor. Our system has a deformable surface on the back of the watch face, allowing the visual scene on screen to extend into 2.5D physical space. This allows the user to watch and feel virtual objects, such as experiencing a ball bouncing against the wrist. We devised two controlled experiments to analyze the influence of tactile display resolution on the illusion of virtual object presence. Our first study revealed that on average, a taxel can render virtual objects between 70% and 138% of its own size without shattering the illusion. From the second study, we found visual and haptic feedback can be separated by 4.5mm to 16.2mm for the tested taxels. Based on the results, we developed a prototype (called RetroShape) with 4×4 10mm taxels using micro servo motors, and demonstrated its unique capability through a set of tactile-enhanced games and videos. A preliminary user evaluation showed that participants welcome RetroShape as a useful addition to existing smartwatch output.

[1]  M. Zigler,et al.  The Tactual Perception of Form , 1926 .

[2]  H. Levitt.  Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[3]  제임스 에프. 크래머.  Force feedback and textures simulating interface device , 1992 .

[4]  Hong Z. Tan,et al.  HUMAN FACTORS FOR THE DESIGN OF FORCE-REFLECTING HAPTIC INTERFACES , 1994 .

[5]  Hiroo Iwata,et al.  Project FEELEX: adding haptic surface to graphics , 2001, SIGGRAPH.

[6]  Alan C. Brady,et al.  Results from a Tactile Array on the Fingertip , 2003 .

[7]  Günther Schmidt,et al.  Multi-fingered tactile feedback from virtual and remote environments , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[8]  Vincent Hayward,et al.  Haptic interfaces and devices , 2004 .

[9]  Lorna M. Brown,et al.  Tactons: Structured Tactile Messages for Non-Visual Information Display , 2004, AUIC.

[10]  Abderrahmane Kheddar,et al.  Tactile interfaces: a state-of-the-art survey , 2004 .

[11]  Yoshinori Kuno,et al.  Multimodal presentation method for a dance training system , 2005, CHI Extended Abstracts.

[12]  Vincent Hayward,et al.  A role for haptics in mobile interaction: initial design using a handheld tactile display prototype , 2006, CHI.

[13]  Ivan Poupyrev,et al.  Actuation and tangible user interfaces: the Vaucanson duck, robots, and shape displays , 2007, TEI.

[14]  Cynthia Breazeal,et al.  TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning , 2007, IEEE Transactions on Robotics.

[15]  Naoki Kawakami,et al.  Proposal for tactile sense presentation that combines electrical and mechanical stimulus , 2007, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07).

[16]  Jérôme Szewczyk,et al.  Tactile Rendering With Shape-Memory-Alloy Pin-Matrix , 2008, IEEE Transactions on Instrumentation and Measurement.

[17]  Martin Buss,et al.  Visual–haptic perception of compliant objects in artificially generated environments , 2008, The Visual Computer.

[18]  Jan O. Borchers,et al.  Tactile motion instructions for physical activities , 2009, CHI.

[19]  Thad Starner,et al.  Mobile gesture interaction using wearable tactile displays , 2009, CHI Extended Abstracts.

[20]  Ali Israr,et al.  TeslaTouch: electrovibration for touch surfaces , 2010, UIST.

[21]  Subhas Chandra Mukhopadhyay,et al.  Wearable and Autonomous Biomedical Devices and Systems for Smart Environment: Issues and Characterization , 2010 .

[22]  Thad Starner,et al.  BuzzWear: alert perception in wearable tactile displays on the wrist , 2010, CHI.

[23]  Vincent Hayward,et al.  Biomechanically Optimized Distributed Tactile Transducer Based on Lateral Skin Deformation , 2010, Int. J. Robotics Res..

[24]  Alois Ferscha,et al.  Tactor placement in wrist worn wearables , 2010, International Symposium on Wearable Computers (ISWC) 2010.

[25]  Sethuraman Panchanathan,et al.  Motor learning using a kinematic-vibrotactile mapping targeting fundamental movements , 2011, MM '11.

[26]  Jérôme Pasquero,et al.  A haptic wristwatch for eyes-free interactions , 2011, CHI.

[27]  Ali Israr,et al.  Tactile brush: drawing on skin with a tactile grid display , 2011, CHI.

[28]  Alex Olwal,et al.  Multimodal motion guidance: techniques for adaptive and dynamic feedback , 2012, ICMI '12.

[29]  Ivan Poupyrev,et al.  AIREAL: tactile gaming experiences in free air , 2013, SIGGRAPH '13.

[30]  Hiroshi Ishii,et al.  PneUI: pneumatically actuated soft composite materials for shape changing interfaces , 2013, UIST.

[31]  Hiroshi Ishii,et al.  inFORM: dynamic physical affordances and constraints through shape and object actuation , 2013, UIST.

[32]  Sriram Subramanian,et al.  UltraHaptics: multi-point mid-air haptic feedback for touch surfaces , 2013, UIST.

[33]  Hiroshi Ishii,et al.  Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration , 2014, UIST.

[34]  H. Olausson,et al.  Touch perceptions across skin sites: differences between sensitivity, direction discrimination and pleasantness , 2014, Front. Behav. Neurosci..

[35]  Li-Wei Chan,et al.  ThirdHand: wearing a robotic arm to experience rich force feedback , 2015, SIGGRAPH Asia Emerging Technologies.

[36]  Hiroshi Ishii,et al.  LineFORM: Actuated Curve Interfaces for Display, Interaction, and Constraint , 2015, UIST.

[37]  Morten Fjeld,et al.  OmniVib: Towards Cross-body Spatiotemporal Vibrotactile Notifications for Mobile Phones , 2015, CHI.

[38]  Geehyuk Lee,et al.  Investigating the Information Transfer Efficiency of a 3x3 Watch-back Tactile Display , 2015, CHI.

[39]  Patrick Baudisch,et al.  Skin Drag Displays: Dragging a Physical Tactor across the User's Skin Produces a Stronger Tactile Stimulus than Vibrotactile , 2015, CHI.

[40]  Li-Wei Chan,et al.  EdgeVib: Effective Alphanumeric Character Output Using a Wrist-Worn Tactile Display , 2016, UIST.

[41]  Ramiro Velazquez,et al.  Wearable Assistive Devices for the Blind , 2016, ArXiv.

[42]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[43]  Woohun Lee,et al.  HapThimble: A Wearable Haptic Device towards Usable Virtual Touch Screen , 2016, CHI.

[44]  Hiroshi Ishii,et al.  Materiable: Rendering Dynamic Material Properties in Response to Direct Physical Touch with Shape Changing Interfaces , 2016, CHI.

[45]  Hiroshi Ishii,et al.  Haptic Edge Display for Mobile Tactile Interaction , 2016, CHI.

[46]  Pattie Maes,et al.  Body Integrated Programmable Joints Interface , 2016, CHI.

[47]  Ryuta Okazaki,et al.  FinGAR: combination of electrical and mechanical stimulation for high-fidelity tactile presentation , 2016, SIGGRAPH Emerging Technologies.

[48]  Liwei Chan,et al.  tactoRing: A Skin-Drag Discrete Display , 2017, CHI.