Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback

We present Omni, a self-contained 3D haptic feedback system that is capable of sensing and actuating an untethered, passive tool containing only a small embedded permanent magnet. Omni enriches AR, VR and desktop applications by providing an active haptic experience using a simple apparatus centered around an electromagnetic base. The spatial haptic capabilities of Omni are enabled by a novel gradient-based method to reconstruct the 3D position of the permanent magnet in midair using the measurements from eight off-the-shelf hall sensors that are integrated into the base. Omni's 3 DoF spherical electromagnet simultaneously exerts dynamic and precise radial and tangential forces in a volumetric space around the device. Since our system is fully integrated, contains no moving parts and requires no external tracking, it is easy and affordable to fabricate. We describe Omni's hardware implementation, our 3D reconstruction algorithm, and evaluate the tracking and actuation performance in depth. Finally, we demonstrate its capabilities via a set of interactive usage scenarios.

[1]  Eyal Ofek,et al.  CapstanCrunch: A Haptic VR Controller with User-supplied Force Feedback , 2019, UIST.

[2]  Velko Vechev,et al.  DextrES: Wearable Haptic Feedback for Grasping in VR via a Thin Form-Factor Electrostatic Brake , 2018, UIST.

[3]  Bing-Yu Chen,et al.  GaussMarbles: Spherical Magnetic Tangibles for Interacting with Portable Physical Constraints , 2016, CHI.

[4]  David Lindlbauer,et al.  Optimal Control for Electromagnetic Haptic Guidance Systems , 2020, UIST.

[5]  Jake J. Abbott,et al.  Omnimagnet: An Omnidirectional Electromagnet for Controlled Dipole-Field Generation , 2014, IEEE Transactions on Magnetics.

[6]  P.-A. Besse,et al.  Tracking system with five degrees of freedom using a 2D-array of Hall sensors and a permanent magnet , 2001 .

[7]  Peter J. Berkelman,et al.  Co-located 3D graphic and haptic display using electromagnetic levitation , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[8]  Jake J. Abbott,et al.  Six principal modes of vibrotactile display via stylus , 2018, 2018 IEEE Haptics Symposium (HAPTICS).

[9]  Sebastian Boring,et al.  Magnetips: Combining Fingertip Tracking and Haptic Feedback for Around-Device Interaction , 2019, CHI.

[10]  D. A. Dunnett.  Classical Electrodynamics , 2020, Nature.

[11]  Wendy Ju,et al.  Origami Desk: integrating technological innovation and human-centric design , 2002, DIS '02.

[12]  K. Kilgore,et al.  Implementation of an implantable joint-angle transducer. , 2002, Journal of rehabilitation research and development.

[13]  Byung-Ju Yi,et al.  Design and Experimental Verification of a 3-DOF Spherical Electromagnetic Brake for Haptic Interface , 2020 .

[14]  Woohun Lee,et al.  HapThimble: A Wearable Haptic Device towards Usable Virtual Touch Screen , 2016, CHI.

[15]  Otmar Hilliges,et al.  Contact-free Nonplanar Haptics with a Spherical Electromagnet , 2020, 2020 IEEE Haptics Symposium (HAPTICS).

[16]  Brendan Iribe.  Oculus Rift를 이용한 체감형 게임 구현 , 2014 .

[17]  F. Clemente,et al.  The myokinetic control interface: tracking implanted magnets as a means for prosthetic control , 2017, Scientific Reports.

[18]  R. E. Stamper,et al.  A Three Degree of Freedom Parallel Manipulator with Only Translational Degrees of Freedom , 1997 .

[19]  Peter Berkelman,et al.  Electromagnetic Position Sensing and Force Feedback for a Magnetic Stylus With an Interactive Display , 2019, IEEE Magnetics Letters.

[20]  Jan O. Borchers,et al.  Madgets: actuating widgets on interactive tabletops , 2010, UIST.

[21]  Yasuaki Kakehi,et al.  dePENd: augmented handwriting system using ferromagnetism of a ballpoint pen , 2013, UIST.

[22]  Kar W. Yung,et al.  An Analytic Solution for the Force Between Two Magnetic Dipoles , 1998 .

[23]  Eyal Ofek,et al.  TORC: A Virtual Reality Controller for In-Hand High-Dexterity Finger Interaction , 2019, CHI.

[24]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[25]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[26]  Eyal Ofek,et al.  CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality , 2018, CHI.

[27]  Daniel J. Wigdor,et al.  Snake Charmer: Physically Enabling Virtual Objects , 2016, TEI.

[28]  Karon E. MacLean,et al.  Harold's purple crayon rendered in haptics: Large-stroke, handheld ballpoint force feedback , 2018, 2018 IEEE Haptics Symposium (HAPTICS).

[29]  Peter Berkelman,et al.  Interactive Rigid-Body Dynamics and Deformable Surface Simulations with Co-Located Maglev Haptic and 3D Graphic Display , 2013 .

[30]  Hakan Gurocak,et al.  Haptic joystick with hybrid actuator using air muscles and spherical MR-brake , 2011 .

[31]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[32]  Velko Vechev,et al.  A Demonstration on Dynamic Drawing Guidance via Electromagnetic Haptic Feedback , 2019, UIST.

[33]  Peter Goldsmith,et al.  A Neural Network Driven Sensor Array for Locating a Permanent Magnet , 2017 .

[34]  Hakan Gurocak,et al.  Spherical Brake with MR Fluid as Multi Degree of Freedom Actuator for Haptics , 2009 .

[35]  Yifei Zhang,et al.  Dexmo: An Inexpensive and Lightweight Mechanical Exoskeleton for Motion Capture and Force Feedback in VR , 2016, CHI.

[36]  Sean White,et al.  Nenya: subtle and eyes-free mobile input with a magnetically-tracked finger ring , 2011, CHI.

[37]  Wolfgang Straßer,et al.  Magnets in motion , 2008, SIGGRAPH Asia '08.

[38]  Ali Israr,et al.  Tactile brush: drawing on skin with a tactile grid display , 2011, CHI.

[39]  Jan O. Borchers,et al.  FingerFlux: near-surface haptic feedback on tabletops , 2011, UIST.

[40]  Philippe C. Cattin,et al.  Assessment of the Functional Rotational Workspace of Different Grasp Type Handles for the lambda.6 Haptic Device* , 2019, 2019 IEEE World Haptics Conference (WHC).

[41]  Tim Ameel,et al.  A Thermal Transient Model of Heat Transfer Within an Omnimagnet , 2017 .

[42]  Ralph L. Hollis,et al.  Lorentz Magnetic Levitation for Haptic Interaction: Device Design, Performance, and Integration with Physical Simulations , 2000, Int. J. Robotics Res..

[43]  P. Berkelman,et al.  DESIGN OF A HEMISPHERICAL MAGNETIC LEVITATION HAPTIC INTERFACE DEVICE , 1999 .

[44]  Abdulmotaleb El-Saddik,et al.  Magnetic Field Control for Haptic Display: System Design and Simulation , 2016, IEEE Access.

[45]  Sean Follmer,et al.  Wolverine: A Wearable Haptic Interface for Grasping in VR , 2016, UIST.

[46]  Masa Ogata,et al.  Magneto-Haptics: Embedding Magnetic Force Feedback for Physical Interactions , 2018, UIST.

[47]  Shwetak N. Patel,et al.  Finexus: Tracking Precise Motions of Multiple Fingertips Using Magnetic Sensing , 2016, CHI.

[48]  H. Seki,et al.  Wearable handwriting input device using magnetic field , 2007, SICE Annual Conference 2007.

[49]  Geehyuk Lee,et al.  Thor's Hammer: An Ungrounded Force Feedback Device Utilizing Propeller-Induced Propulsive Force , 2018, CHI Extended Abstracts.

[50]  Bing-Yu Chen,et al.  GaussSense: attachable stylus sensing using magnetic sensor grid , 2012, UIST '12.

[51]  Ellen Yi-Luen Do,et al.  Mechamagnets: Designing and Fabricating Haptic and Functional Physical Inputs with Embedded Magnets , 2019, TEI.

[52]  Claudio Pacchierotti,et al.  Towards Wearability in Fingertip Haptics: A 3-DoF Wearable Device for Cutaneous Force Feedback , 2013, IEEE Transactions on Haptics.

[53]  Islam S. M. Khalil,et al.  Magnetic Localization for an Electromagnetic-Based Haptic Interface , 2019, IEEE Magnetics Letters.

[54]  Jake J. Abbott,et al.  Factors affecting the design of untethered magnetic haptic interfaces , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[55]  Hugh M. Herr,et al.  Low-Latency Tracking of Multiple Permanent Magnets , 2019, IEEE Sensors Journal.

[56]  Bin Liu,et al.  A 2-DOF MR actuator joystick for virtual reality applications , 2007 .

[57]  D. Maynes-Aminzade,et al.  The actuated workbench: computer-controlled actuation in tabletop tangible interfaces , 2003, ACM Trans. Graph..

[58]  Hiroyuki Shinoda,et al.  Noncontact Tactile Display Based on Radiation Pressure of Airborne Ultrasound , 2010, IEEE Transactions on Haptics.