Magnetips: Combining Fingertip Tracking and Haptic Feedback for Around-Device Interaction

Around-device interaction methods expand the available interaction space for mobile devices; however, there is currently no way to simultaneously track a user's input and provide haptic feedback at the tracked point away from the device. We present Magnetips, a simple, mobile solution for around-device tracking and mid-air haptic feedback. Magnetips combines magnetic tracking and electromagnetic feedback that works regardless of visual occlusion, through most common materials, and at a size that allows for integration with mobile devices. We demonstrate: (1) high-frequency around-device tracking and haptic feedback; (2) the accuracy and range of our tracking solution which corrects for the effects of geomagnetism, necessary for enabling mobile use; and (3) guidelines for maximising strength of haptic feedback, given a desired tracking frequency. We present technical and usability evaluations of our prototype, and demonstrate four example applications of its use.

[1]  David Coyle,et al.  Extending interaction for smart watches: enabling bimanual around device control , 2014, CHI Extended Abstracts.

[2]  Mathieu Le Goc,et al.  A low-cost transparent electric field sensor for 3d interaction on mobile devices , 2014, CHI.

[3]  Sriram Subramanian,et al.  Sparkle: Hover Feedback with Touchable Electric Arcs , 2017, CHI.

[4]  Itiro Siio,et al.  MagNail: user interaction with smart device through magnet attached to fingernail , 2015, UbiComp/ISWC Adjunct.

[5]  Sriram Subramanian,et al.  Perception of ultrasonic haptic feedback on the hand: localisation and apparent motion , 2014, CHI.

[6]  Shahram Izadi,et al.  SideSight: multi-"touch" interaction around small devices , 2008, UIST '08.

[7]  Xiang 'Anthony' Chen,et al.  The fat thumb: using the thumb's contact size for single-handed mobile interaction , 2012, Mobile HCI.

[8]  Stefan Schneegaß,et al.  GestureSleeve: using touch sensitive fabrics for gestural input on the forearm for controlling smartwatches , 2016, SEMWEB.

[9]  Patrick Baudisch,et al.  Back-of-device interaction allows creating very small touch devices , 2009, CHI.

[10]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[11]  R. Schmidt.  Fundamentals of Sensory Physiology , 1986, Springer Study Edition.

[12]  Gierad Laput,et al.  AuraSense: Enabling Expressive Around-Smartwatch Interactions with Electric Field Sensing , 2016, UIST.

[13]  Kenton O'Hara,et al.  Pre-Touch Sensing for Mobile Interaction , 2016, CHI.

[14]  Sebastian Boring,et al.  WatchSense: On- and Above-Skin Input Sensing through a Wearable Depth Sensor , 2017, CHI.

[15]  Sebastian Madgwick,et al.  Estimation of IMU and MARG orientation using a gradient descent algorithm , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[16]  Joanna Bergstrom-Lehtovirta,et al.  Placing and Recalling Virtual Items on the Skin , 2017, CHI.

[17]  Ian Oakley,et al.  Interaction on the edge: offset sensing for small devices , 2014, CHI.

[18]  Sriram Subramanian,et al.  UltraHaptics: multi-point mid-air haptic feedback for touch surfaces , 2013, UIST.

[19]  Yuriko Suzuki,et al.  Air jet driven force feedback in virtual reality , 2005, IEEE Computer Graphics and Applications.

[20]  Peter Fröhlich,et al.  Markerless visual fingertip detection for natural mobile device interaction , 2011, Mobile HCI.

[21]  Jan O. Borchers,et al.  FingerFlux: near-surface haptic feedback on tabletops , 2011, UIST.

[22]  Chris Harrison,et al.  Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices , 2009, UIST '09.

[23]  Sean White,et al.  uTrack: 3D input using two magnetic sensors , 2013, UIST.

[24]  Ali Israr,et al.  AIREAL: interactive tactile experiences in free air , 2013, ACM Trans. Graph..

[25]  Radu-Daniel Vatavu,et al.  Digital vibrons: understanding users' perceptions of interacting with invisible, zero-weight matter , 2016, MobileHCI.

[26]  Hamed Ketabdar,et al.  Towards using embedded magnetic field sensor for around mobile device 3D interaction , 2010, Mobile HCI.

[27]  Sean White,et al.  Nenya: subtle and eyes-free mobile input with a magnetically-tracked finger ring , 2011, CHI.

[28]  Stephen A. Brewster,et al.  Tactile Feedback for Above-Device Gesture Interfaces: Adding Touch to Touchless Interactions , 2014, ICMI.

[29]  James R. Eagan,et al.  Watchit: simple gestures and eyes-free interaction for wristwatches and bracelets , 2013, CHI.

[30]  M. Hollins,et al.  The vibrations of texture , 2003, Somatosensory & motor research.

[31]  Michita Imai,et al.  SkinWatch: skin gesture interaction for smart watch , 2015, AH.

[32]  Kasper Hornbæk,et al.  Generating Haptic Textures with a Vibrotactile Actuator , 2017, CHI.

[33]  Roel Vertegaal,et al.  Effects of Display Sizes on a Scrolling Task using a Cylindrical Smartwatch , 2015, MobileHCI Adjunct.

[34]  Shwetak N. Patel,et al.  Finexus: Tracking Precise Motions of Multiple Fingertips Using Magnetic Sensing , 2016, CHI.

[35]  H. Seki,et al.  Wearable handwriting input device using magnetic field , 2007, SICE Annual Conference 2007.