Adaptic: A Shape Changing Prop with Haptic Retargeting

We present Adaptic, a novel ”hybrid” active/passive haptic device that can change shape to act as a proxy for a range of virtual objects in VR. We use Adaptic with haptic retargeting to redirect the user’s hand to provide haptic feedback for several virtual objects in arm’s reach using only a single prop. To evaluate the effectiveness of Adaptic with haptic retargeting, we conducted a within-subjects experiment employing a docking task to compare Adaptic to non-matching proxy objects (i.e., Styrofoam balls) and matching shape props. In our study, Adaptic sat on a desk in front of the user and changed shapes between grasps, to provide matching tactile feedback for various virtual objects placed in different virtual locations. Results indicate that the illusion was convincing: users felt they were manipulating several virtual objects in different virtual locations with a single Adaptic device. Docking performance (completion time and accuracy) with Adaptic was comparable to props without haptic retargeting.

[1]  Mary C. Whitton,et al.  15 Years of Research on Redirected Walking in Immersive Virtual Environments , 2018, IEEE Computer Graphics and Applications.

[2]  Antonio Krüger,et al.  Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality , 2017, IEEE Transactions on Visualization and Computer Graphics.

[3]  Simon Richir,et al.  Influence of control/display ratio on the perception of mass of manipulated objects in virtual environments , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[4]  Bing-Yu Chen,et al.  ElastOscillation: 3D Multilevel Force Feedback for Damped Oscillation on VR Controllers , 2020, CHI.

[5]  Mary C. Whitton,et al.  Passive haptics significantly enhances virtual environments , 2001 .

[6]  I ROCK,et al.  Vision and Touch: An Experimentally Created Conflict between the Two Senses , 1964, Science.

[7]  Eyal Ofek,et al.  TORC: A Virtual Reality Controller for In-Hand High-Dexterity Finger Interaction , 2019, CHI.

[8]  Nikolaus F. Troje,et al.  Paper windows: interaction techniques for digital paper , 2005, CHI.

[9]  Josef B. Spjut,et al.  A variable shape and variable stiffness controller for haptic virtual interactions , 2018, 2018 IEEE International Conference on Soft Robotics (RoboSoft).

[10]  Makoto Sato,et al.  Development of string-based multi-finger haptic interface SPIDAR-MF , 2013, 2013 23rd International Conference on Artificial Reality and Telexistence (ICAT).

[11]  Andreas L. Lohse,et al.  Leveraging Change Blindness for Haptic Remapping in Virtual Environments , 2019, 2019 IEEE 5th Workshop on Everyday Virtual Reality (WEVR).

[12]  Takuji Narumi,et al.  Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering based on Computational Perception Model , 2019, CHI.

[13]  Sriram Subramanian,et al.  Is my phone alive?: a large-scale study of shape change in handheld devices using videos , 2014, CHI.

[14]  Pierre Dragicevic,et al.  Zooids: Building Blocks for Swarm User Interfaces , 2016, UIST.

[15]  Hiroshi Ishii,et al.  Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices , 2012, UIST.

[16]  Raimund Dachselt,et al.  Foldable3D: Interacting with 3D Content Using Dual-Display Devices , 2016, ISS.

[17]  Maud Marchal,et al.  Elastic-Arm: Human-scale passive haptic feedback for augmenting interaction and perception in virtual environments , 2015, 2015 IEEE Virtual Reality (VR).

[18]  Sean Follmer,et al.  Wolverine: A wearable haptic interface for grasping in virtual reality , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  David Kim,et al.  FlexSense: a transparent self-sensing deformable surface , 2014, UIST.

[20]  Antonio Krüger,et al.  Combining Dynamic Passive Haptics and Haptic Retargeting for Enhanced Haptic Feedback in Virtual Reality , 2021, IEEE Transactions on Visualization and Computer Graphics.

[21]  Roel Vertegaal,et al.  PaperPhone: understanding the use of bend gestures in mobile devices with flexible electronic paper displays , 2011, CHI.

[22]  Nicholas Chen,et al.  Navigation techniques for dual-display e-book readers , 2008, CHI.

[23]  Hiroo Iwata,et al.  Haptic interfaces , 2002 .

[24]  Sharif Razzaque,et al.  Redirected Walking , 2001, Eurographics.

[25]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[26]  Gerard Jounghyun Kim,et al.  Effects of sizes and shapes of props in tangible augmented reality , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[27]  Youngwoo Park,et al.  The Trial of Bendi in a Coffeehouse: Use of a Shape-Changing Device for a Tactile-Visual Phone Conversation , 2015, CHI.

[28]  Eyal Ofek,et al.  Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop , 2017, CHI.

[29]  Laurence Nigay,et al.  EXHI-bit: a mechanical structure for prototyping EXpandable handheld interfaces , 2017, MobileHCI.

[30]  J.C. Perry,et al.  Upper-Limb Powered Exoskeleton Design , 2007, IEEE/ASME Transactions on Mechatronics.

[31]  Kening Zhu,et al.  HapTwist: creating interactive haptic proxies in virtual reality using low-cost twistable artefacts , 2018, SIGGRAPH 2018.

[32]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[33]  Fabian Hemmert,et al.  Shape-changing mobiles: tapering in two-dimensional deformational displays in mobile phones , 2010, CHI Extended Abstracts.

[34]  Pedro Lopes,et al.  Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation , 2017, CHI.

[35]  Takuji Narumi,et al.  Computational design of hand-held VR controllers using haptic shape illusion , 2017, VRST.

[36]  Joseph A. Paradiso,et al.  ChainFORM: A Linear Integrated Modular Hardware System for Shape Changing Interfaces , 2016, UIST.

[37]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[38]  Roel Vertegaal,et al.  PaperFold: Evaluating Shape Changes for Viewport Transformations in Foldable Thin-Film Display Devices , 2015, Tangible and Embedded Interaction.

[39]  Robert J. Teather,et al.  Haptobend: shape-changing passive haptic feedback in virtual reality , 2017, SUI.

[40]  Lawrence H. Kim,et al.  Robotic Assembly of Haptic Proxy Objects for Tangible Interaction and Virtual Reality , 2017, ISS.

[41]  June-Hao Hou,et al.  Pneu-Multi-Tools: Auto-Folding and Multi-Shapes Interface by Pneumatics in Virtual Reality , 2019, UIST.

[42]  Antonio Krüger,et al.  Drag:on: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift , 2019, CHI.

[43]  Johannes Schöning,et al.  Paddle: highly deformable mobile devices with physical controls , 2014, CHI Extended Abstracts.

[44]  Robert W. Lindeman,et al.  Towards usable VR: an empirical study of user interfaces for immersive virtual environments , 1999, CHI '99.

[45]  H Tanaka,et al.  Programmable matter by folding , 2010, Proceedings of the National Academy of Sciences.

[46]  Markus Löchtefeld,et al.  Morphees: toward high "shape resolution" in self-actuated flexible mobile devices , 2013, CHI.

[47]  Hans-Werner Gellersen,et al.  Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences , 2015, CHI.

[48]  Antonio Krüger,et al.  Visuo-haptic Illusions for Linear Translation and Stretching using Physical Proxies in Virtual Reality , 2021, CHI.

[49]  Pattie Maes,et al.  Flexpad: highly flexible bending interactions for projected handheld displays , 2013, CHI.

[50]  Cathy Fang,et al.  Wireality: Enabling Complex Tangible Geometries in Virtual Reality with Worn Multi-String Haptics , 2020, CHI.

[51]  Patrick Baudisch,et al.  iTurk: Turning Passive Haptics into Active Haptics by Making Users Reconfigure Props in Virtual Reality , 2018, CHI.

[52]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[53]  Kai Kunze,et al.  FingerFlex: Shape Memory Alloy-based Actuation on Fingers for Kinesthetic Haptic Feedback , 2020, MUM.

[54]  Miguel Bruns Alonso,et al.  Balancing User and System Control in Shape-Changing Interfaces: a Designerly Exploration , 2016, Tangible and Embedded Interaction.

[55]  Hunter G. Hoffman,et al.  Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[56]  Takuji Narumi,et al.  Modifying an identified curved surface shape using pseudo-haptic effect , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[57]  M. Alexa,et al.  Combining Shape-Changing Interfaces and Spatial Augmented Reality Enables Extended Object Appearance , 2016, CHI.

[58]  Eyal Ofek,et al.  CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality , 2018, CHI.

[59]  Mary C. Whitton,et al.  Redirected Touching: Training and adaptation in warped virtual spaces , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[60]  Kyle Johnsen,et al.  Performance Benefits of High-Fidelity Passive Haptic Feedback in Virtual Reality Training , 2018, SUI.

[61]  Roel Vertegaal,et al.  Towards more paper-like input: flexible input devices for foldable interaction styles , 2008, UIST '08.

[62]  Alex Olwal,et al.  Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality , 2017, UIST.

[63]  Anatole Lécuyer,et al.  Reconfigurable tangible devices for 3D virtual object manipulation by single or multiple users , 2010, VRST '10.

[64]  Tobias Isenberg,et al.  Mouse, Tactile, and Tangible Input for 3D Manipulation , 2016, CHI.

[65]  Jürgen Steimle,et al.  FoldMe: interacting with double-sided foldable displays , 2012, Tangible and Embedded Interaction.

[66]  Audrey Girouard,et al.  Fabricating Bendy: Design and Development of Deformable Prototypes , 2014, IEEE Pervasive Computing.

[67]  Sean Follmer,et al.  shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction , 2018, CHI.

[68]  Modris Greitans,et al.  Acceleration and Magnetic Sensor Network for Shape Sensing , 2016, IEEE Sensors Journal.

[69]  Eyal Ofek,et al.  Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences , 2016, CHI.

[70]  Da Young Ju,et al.  Emotional Interaction and Nofitication of Flexible Handheld Devices , 2015, CHI Extended Abstracts.

[71]  Da-Yuan Huang,et al.  PuPoP: Pop-up Prop on Palm for Virtual Reality , 2018, UIST.

[72]  Hiroshi Ishii,et al.  LineFORM: Actuated Curve Interfaces for Display, Interaction, and Constraint , 2015, UIST.

[73]  Ravin Balakrishnan,et al.  Codex: a dual screen tablet computer , 2009, CHI.

[74]  Michael R. M. Jenkin,et al.  Evaluating haptic feedback in virtual environments using ISO 9241–9 , 2010, 2010 IEEE Virtual Reality Conference (VR).

[75]  Antonio Krüger,et al.  Blink-Suppressed Hand Redirection , 2021, 2021 IEEE Virtual Reality and 3D User Interfaces (VR).

[76]  Roel Vertegaal,et al.  An Evaluation of Shape Changes for Conveying Emotions , 2016, CHI.

[77]  Anne C. Sittig,et al.  The precision of proprioceptive position sense , 1998, Experimental Brain Research.

[78]  Takuji Narumi,et al.  ShapeSense: a 2D shape rendering VR device with moving surfaces that controls mass properties and air resistance , 2019, SIGGRAPH Emerging Technologies.

[79]  Roel Vertegaal,et al.  MorePhone: a study of actuated shape deformations for flexible thin-film smartphone notifications , 2013, CHI.

[80]  Jaeyoung Park,et al.  Haptic Feedback to the Palm and Fingers for Improved Tactile Perception of Large Objects , 2018, UIST.