Tactile hand motion and pose guidance for 3D interaction

We present a novel forearm-and-glove tactile interface that can enhance 3D interaction by guiding hand motor planning and coordination. In particular, we aim to improve hand motion and pose actions related to selection and manipulation tasks. Through our user studies, we illustrate how tactile patterns can guide the user, by triggering hand pose and motion changes, for example to grasp (select) and manipulate (move) an object. We discuss the potential and limitations of the interface, and outline future work.

[1]  Pedro Lopes,et al.  Muscle-plotter: An Interactive System based on Electrical Muscle Stimulation that Produces Spatial Output , 2016, UIST.

[2]  Yi Jing,et al.  ERRATUM: Tactile Feedback Display with Spatial and Temporal Resolutions , 2013, Scientific Reports.

[3]  Richard A. Schmidt,et al.  Motor Learning and Performance , 1991 .

[4]  Qibing Pei,et al.  Tactile Feedback Display with Spatial and Temporal Resolutions , 2013, Scientific reports.

[5]  Paul Green,et al.  The Rapid Development of User Interfaces: Experience with the Wizard of OZ Method , 1985 .

[6]  Jan O. Borchers,et al.  Tactile motion instructions for physical activities , 2009, CHI.

[7]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[8]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[9]  V. Maheshwari,et al.  Tactile devices to sense touch on a par with a human finger. , 2008, Angewandte Chemie.

[10]  Jianwei Zhang,et al.  Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task , 2016, CAAI Trans. Intell. Technol..

[11]  Holger Regenbrecht,et al.  Virtual reality aided assembly with directional vibro-tactile feedback , 2005, GRAPHITE '05.

[12]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[13]  W.J. Tompkins,et al.  Electrotactile and vibrotactile displays for sensory substitution systems , 1991, IEEE Transactions on Biomedical Engineering.

[14]  José Pascual Molina,et al.  Identifying Virtual 3D Geometric Shapes with a Vibrotactile Glove , 2016, IEEE Computer Graphics and Applications.

[15]  Dieter Drobny,et al.  Learning basic dance choreographies with different augmented feedback modalities , 2010, CHI Extended Abstracts.

[16]  Dinesh K. Pai,et al.  Multisensory Interaction: Real and Virtual , 2003, ISRR.

[17]  Alex Olwal,et al.  Multimodal motion guidance: techniques for adaptive and dynamic feedback , 2012, ICMI '12.

[18]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[19]  Thorsten A. Kern,et al.  Engineering Haptic Devices: A Beginner's Guide , 2014 .

[20]  Sethuraman Panchanathan,et al.  MOVeMENT: A framework for systematically mapping vibrotactile stimulations to fundamental body movements , 2010, 2010 IEEE International Symposium on Haptic Audio Visual Environments and Games.

[21]  Ferran Argelaguet,et al.  See-through techniques for referential awareness in collaborative virtual reality , 2011, Int. J. Hum. Comput. Stud..

[22]  Paul R. Schrater,et al.  Within- and Cross-Modal Distance Information Disambiguate Visual Size-Change Perception , 2010, PLoS Comput. Biol..

[23]  Antonio Frisoli,et al.  Vibrotactile perception assessment for a rowing training system , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[24]  Cynthia Breazeal,et al.  TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning , 2007, IEEE Transactions on Robotics.

[25]  Doug A. Bowman,et al.  Evaluating Display Fidelity and Interaction Fidelity in a Virtual Reality Game , 2012, IEEE Transactions on Visualization and Computer Graphics.

[26]  Yi-Ju Chung,et al.  Motion Guidance Sleeve: Guiding the Forearm Rotation through External Artificial Muscles , 2016, CHI.

[27]  Norman I. Badler,et al.  Virtual Training via Vibrotactile Arrays , 2008, PRESENCE: Teleoperators and Virtual Environments.

[28]  Ying Zheng,et al.  A vibrotactile feedback approach to posture guidance , 2010, 2010 IEEE Haptics Symposium.

[29]  Johannes Schöning,et al.  Audio-tactile proximity feedback for enhancing 3D manipulation , 2018, VRST.

[30]  Tobias Höllerer,et al.  Interactive tools for virtual x-ray vision in mobile augmented reality , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[31]  Julie A. Jacko,et al.  Multimodal feedback: establishing a performance baseline for improved access by individuals with visual impairments , 2002, Assets '02.

[32]  C. Spence,et al.  Multisensory Integration: Maintaining the Perception of Synchrony , 2003, Current Biology.

[33]  Jun Rekimoto,et al.  PossessedHand: techniques for controlling human hands using electrical muscles stimuli , 2011, CHI.

[34]  Manfred Huber,et al.  Enhanced targeting in a haptic user interface for the physically disabled using a force feedback mouse , 2007 .

[35]  Dieter Schmalstieg,et al.  Tactylus, a pen-input device exploring audiotactile sensory binding , 2006, VRST '06.

[36]  R. Ammons.  Effects of Knowledge of Performance: A Survey and Tentative Theoretical Formulation , 1956 .

[37]  Steffi Beckhaus,et al.  How to not hit a virtual wall: aural spatial awareness for collision avoidance in virtual environments , 2011, AM '11.

[38]  Norman I. Badler,et al.  A taxonomy and comparison of haptic actions for disassembly tasks , 2003, IEEE Virtual Reality, 2003. Proceedings..

[39]  Ryuta Okazaki,et al.  HALUX: projection-based interactive skin for digital sports , 2016, SIGGRAPH Emerging Technologies.

[40]  Mark A. Symmons,et al.  Cognitive Load Can Explain Differences in Active and Passive Touch , 2012, EuroHaptics.

[41]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[42]  Steffi Beckhaus,et al.  CubicalPath-dynamic potential fields for guided exploration in virtual environments , 2000, Proceedings the Eighth Pacific Conference on Computer Graphics and Applications.

[43]  Trevor C. Chen,et al.  Muscle damage induced by electrical stimulation , 2011, European Journal of Applied Physiology.

[44]  Robert W. Lindeman,et al.  On Your Feet!: Enhancing Vection in Leaning-Based Interfaces through Multisensory Stimuli , 2016, SUI.

[45]  Danielle Levac,et al.  Motor Learning and Virtual Reality , 2014 .

[46]  Paulo Gallotti,et al.  v-Glove: A 3D Virtual Touch Interface , 2011, 2011 XIII Symposium on Virtual Reality.

[47]  J. Blake,et al.  Haptic Glove With MR Brakes for Virtual Reality , 2009, IEEE/ASME Transactions on Mechatronics.

[48]  Gesche Joost,et al.  Mobile Lorm Glove: introducing a communication device for deaf-blind people , 2012, TEI.

[49]  Erin Piateski,et al.  Vibrotactile pattern recognition on the arm and torso , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[50]  Jens Herder,et al.  Vibrotactile pitfalls: arm guidance for moderators in virtual TV studios , 2010, Humans and Computers.

[51]  Grigore C. Burdea,et al.  Force and Touch Feedback for Virtual Reality , 1996 .

[52]  Kouta Minamizawa,et al.  Haptic telexistence , 2007, SIGGRAPH '07.

[53]  Ernst Kruijff,et al.  Designed emotions: challenges and potential methodologies for improving multisensory cues to enhance user engagement in immersive systems , 2017, The Visual Computer.

[54]  C. Cao,et al.  Role of Haptic Feedback and Cognitive Load in Surgical Skill Acquisition , 2007 .

[55]  Thorsten A. Kern,et al.  Engineering Haptic Devices , 2014, Springer Series on Touch and Haptic Systems.

[56]  Katherine J. Kuchenbecker,et al.  Evaluation of Tactile Feedback Methods for Wrist Rotation Guidance , 2012, IEEE Transactions on Haptics.

[57]  R. Johansson,et al.  Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin. , 1979, The Journal of physiology.

[58]  Katherine J. Kuchenbecker,et al.  Lessons in using vibrotactile feedback to guide fast arm motions , 2011, 2011 IEEE World Haptics Conference.