PoCoPo: Handheld Pin-based Shape Display for Haptic Rendering in Virtual Reality

We introduce PoCoPo, the first handheld pin-based shape display that can render various 2.5D shapes in hand in realtime. We designed the display small enough for a user to hold it in hand and carry it around, thereby enhancing the haptic experiences in a virtual environment. PoCoPo has 18 motor-driven pins on both sides of a cuboid, providing the sensation of skin contact on the user's palm and fingers. We conducted two user studies to understand the capability of PoCoPo. The first study showed that the participants were generally successful in distinguishing the shapes rendered by PoCoPo with an average success rate of 88.5%. In the second study, we investigated the acceptable visual size of a virtual object when PoCoPo rendered a physical object of a certain size. The result led to a better understanding of the acceptable differences between the perceptions of visual size and haptic size.

[1]  Hiroshi Ishii,et al.  Haptic Edge Display for Mobile Tactile Interaction , 2016, CHI.

[2]  Takuji Narumi,et al.  ShapeSense: a 2D shape rendering VR device with moving surfaces that controls mass properties and air resistance , 2019, SIGGRAPH Emerging Technologies.

[3]  Jun Rekimoto,et al.  Lumen: interactive visual and shape display for calm computing , 2004, SIGGRAPH '04.

[4]  I ROCK,et al.  Vision and Touch: An Experimentally Created Conflict between the Two Senses , 1964, Science.

[5]  Da-Yuan Huang,et al.  PuPoP: Pop-up Prop on Palm for Virtual Reality , 2018, UIST.

[6]  Sean Follmer,et al.  Electrostatic adhesive brakes for high spatial resolution refreshable 2.5D tactile shape displays , 2018, 2018 IEEE Haptics Symposium (HAPTICS).

[7]  Naoki Kawakami,et al.  An Encounter-Type Multi-Fingered Master Hand Using Circuitous Joints , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[8]  Hiroo Iwata,et al.  Project FEELEX: adding haptic surface to graphics , 2001, SIGGRAPH.

[9]  Hiroshi Ishii,et al.  inFORM: dynamic physical affordances and constraints through shape and object actuation , 2013, UIST.

[10]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[11]  Hiroshi Ishii,et al.  Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration , 2014, UIST.

[12]  Eyal Ofek,et al.  TORC: A Virtual Reality Controller for In-Hand High-Dexterity Finger Interaction , 2019, CHI.

[13]  Lawrence H. Kim,et al.  Robotic Assembly of Haptic Proxy Objects for Tangible Interaction and Virtual Reality , 2017, ISS.

[14]  Eyal Ofek,et al.  Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences , 2016, CHI.

[15]  Antonio Krüger,et al.  Drag:on: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift , 2019, CHI.

[16]  Hiroshi Ishii,et al.  Direct and gestural interaction with relief: a 2.5D shape display , 2011, UIST '11.

[17]  K O Johnson,et al.  The limit of tactile spatial resolution in humans , 1994, Neurology.

[18]  Eyal Ofek,et al.  CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality , 2018, CHI.

[19]  Yoshihiro Kawahara,et al.  Dynablock: Dynamic 3D Printing for Instant and Reconstructable Shape Formation , 2018, UIST.

[20]  Peter B. Shull,et al.  HaptiVec: Presenting Haptic Feedback Vectors in Handheld Controllers using Embedded Tactile Pin Arrays , 2019, CHI.

[21]  Takuji Narumi,et al.  Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering based on Computational Perception Model , 2019, CHI.

[22]  Aryan Saini,et al.  VirtualBricks: Exploring a Scalable, Modular Toolkit for Enabling Physical Manipulation in VR , 2019, CHI.

[23]  Takuji Narumi,et al.  Modifying an identified curved surface shape using pseudo-haptic effect , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[24]  Eyal Ofek,et al.  Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller , 2018, CHI.

[25]  S.H. Winter,et al.  Use of Magnetorheological Fluid in a Force Feedback Glove , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Sean Follmer,et al.  Visuo-Haptic Illusions for Improving the Perceived Performance of Shape Displays , 2018, CHI.

[27]  Sean Follmer,et al.  Wolverine: A Wearable Haptic Interface for Grasping in VR , 2016, UIST.

[28]  Lynette A. Jones,et al.  Tactile and Haptic Illusions , 2011, IEEE Transactions on Haptics.

[29]  Antonio Krüger,et al.  Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality , 2017, IEEE Transactions on Visualization and Computer Graphics.

[30]  Da-Yuan Huang,et al.  RetroShape: Leveraging Rear-Surface Shape Displays for 2.5D Interaction on Smartwatches , 2017, UIST.

[31]  Ryuta Okazaki,et al.  FinGAR: combination of electrical and mechanical stimulation for high-fidelity tactile presentation , 2016, SIGGRAPH Emerging Technologies.

[32]  Pedro Lopes,et al.  Adding Force Feedback to Mixed Reality Experiences and Games using Electrical Muscle Stimulation , 2018, CHI.

[33]  Gi-Hun Yang,et al.  SaLT: Small and lightweight tactile display using ultrasonic actuators , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.

[34]  Yifei Zhang,et al.  Dexmo: An Inexpensive and Lightweight Mechanical Exoskeleton for Motion Capture and Force Feedback in VR , 2016, CHI.

[35]  Xing-Dong Yang,et al.  Aero-plane: A Handheld Force-Feedback Device that Renders Weight Motion Illusion on a Virtual 2D Plane , 2019, UIST.

[36]  Daniel J. Wigdor,et al.  Annexing Reality: Enabling Opportunistic Use of Everyday Objects as Tangible Proxies in Augmented Reality , 2016, CHI.

[37]  Eyal Ofek,et al.  Haptic Links: Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation , 2018, CHI.

[38]  Geehyuk Lee,et al.  Thor's Hammer: An Ungrounded Force Feedback Device Utilizing Propeller-Induced Propulsive Force , 2018, CHI.

[39]  Takuji Narumi,et al.  Computational design of hand-held VR controllers using haptic shape illusion , 2017, VRST.

[40]  Michitaka Hirose,et al.  Simulation and presentation of curved surface in virtual reality environment through surface display , 1995, Proceedings Virtual Reality Annual International Symposium '95.

[41]  P. Walker,et al.  The brightness-weight illusion. , 2010, Experimental psychology.

[42]  Hiroshi Ishii,et al.  Sublimate: state-changing virtual and physical rendering to augment interaction with shape displays , 2013, CHI.

[43]  K O Johnson,et al.  Tactile spatial resolution. I. Two-point discrimination, gap detection, grating resolution, and letter recognition. , 1981, Journal of neurophysiology.

[44]  Sean Follmer,et al.  shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction , 2018, CHI.

[45]  Takuji Narumi,et al.  PaCaPa: A Handheld VR Device for Rendering Size, Shape, and Stiffness of Virtual Objects in Tool-based Interactions , 2019, CHI.

[46]  Takuji Narumi,et al.  Virtual Hand Realism Affects Object Size Perception in Body-Based Scaling , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[47]  Hiroshi Ishii,et al.  Materiable: Rendering Dynamic Material Properties in Response to Direct Physical Touch with Shape Changing Interfaces , 2016, CHI.

[48]  Sean Follmer,et al.  Applications of Switchable Permanent Magnetic Actuators in Shape Change and Tactile Display , 2016, UIST.

[49]  Hiroshi Ishii,et al.  Kinetic Blocks: Actuated Constructive Assembly for Interaction and Display , 2015, UIST.

[50]  Makoto Shimojo,et al.  Human shape recognition performance for 3D tactile display , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[51]  Hiroshi Ishii,et al.  AnimaStage: Hands-on Animated Craft on Pin-based Shape Displays , 2017, Conference on Designing Interactive Systems.

[52]  Hiroshi Ishii,et al.  inFORCE: Bi-directional `Force' Shape Display for Haptic Interaction , 2019, TEI.

[53]  Alex Olwal,et al.  Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality , 2017, UIST.

[54]  Christopher R. Wagner,et al.  A tactile shape display using RC servomotors , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[55]  Pedro Lopes,et al.  Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation , 2015, UIST.

[56]  Allison M. Okamura,et al.  Fingertip Tactile Devices for Virtual Object Manipulation and Exploration , 2017, CHI.

[57]  Kening Zhu,et al.  HapTwist: creating interactive haptic proxies in virtual reality using low-cost twistable artefacts , 2018, SIGGRAPH 2018.