Active PinScreen: Exploring Spatio-Temporal Tactile Feedbackfor Multi-Finger Interaction

Multiple fingers are often used for efficient interaction with handheld computing devices. Currently, any tactile feedback provided is felt on the finger pad or the palm with coarse granularity. In contrast, we present a new tactile feedback technique, Active PinScreen, that applies localised stimuli on multiple fingers with fine spatial and temporal resolution. The tactile screen uses an array of solenoid-actuated magnetic pins with millimetre scale form-factor which could be deployed for back-of-device handheld use without instrumenting the user. As well as presenting a detailed description of the prototype, we provide the potential design configurations and the applications of the Active PinScreen and evaluate the human factors of tactile interaction with multiple fingers in a controlled user evaluation. The results of our study show a high recognition rate for directional and patterned stimulation across different grip orientations as well as within- and between- fingers. We end the paper with a discussion of our main findings, limitations in the current design and directions for future work.

[1]  Kenton O'Hara,et al.  Pre-Touch Sensing for Mobile Interaction , 2016, CHI.

[2]  J. Craig,et al.  A comparison of tactile spatial sensitivity on the palm and fingerpad , 2001, Perception & psychophysics.

[3]  Niels Henze,et al.  Investigating the feasibility of finger identification on capacitive touchscreens using deep learning , 2019, IUI.

[4]  Vincent Hayward,et al.  A role for haptics in mobile interaction: initial design using a handheld tactile display prototype , 2006, CHI.

[5]  Lorna M. Brown,et al.  Tactons: Structured Tactile Messages for Non-Visual Information Display , 2004, AUIC.

[6]  Hiroshi Ishii,et al.  Haptic Edge Display for Mobile Tactile Interaction , 2016, CHI.

[7]  Gi-Hun Yang,et al.  Quantitative tactile display device with pin-array type tactile feedback and thermal feedback , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[8]  Gerhard Weber,et al.  A tactile windowing system for blind users , 2010, ASSETS '10.

[9]  Geehyuk Lee,et al.  PinPad: Touchpad Interaction with Fast and High-Resolution Tactile Output , 2017, CHI.

[10]  Daniel P. Siewiorek,et al.  Customizable 3D Printed Tactile Maps as Interactive Overlays , 2016, ASSETS.

[11]  Morten Fjeld,et al.  OmniVib: Towards Cross-body Spatiotemporal Vibrotactile Notifications for Mobile Phones , 2015, CHI.

[12]  Hiroshi Ishii,et al.  inFORM: dynamic physical affordances and constraints through shape and object actuation , 2013, UIST.

[13]  Blake Hannaford,et al.  Guest Editorial Special Issue on Biorobotics , 2008, IEEE Transactions on Robotics.

[14]  James A. Landay,et al.  ActiVibe: Design and Evaluation of Vibrations for Progress Monitoring , 2016, CHI.

[15]  Koji Yatani,et al.  SpaceSense: representing geographical information to visually impaired people using spatial tactile feedback , 2012, CHI.

[16]  Stephen A. Brewster,et al.  Investigating the effectiveness of tactile feedback for mobile touchscreens , 2008, CHI.

[17]  Tae-Heon Yang,et al.  Tiny Feel: A New Miniature Tactile Module Using Elastic and Electromagnetic Force for Mobile Devices , 2010, IEICE Trans. Inf. Syst..

[18]  Woohun Lee,et al.  HapCube: A Wearable Tactile Device to Provide Tangential and Normal Pseudo-Force Feedback on a Fingertip , 2018, CHI.

[19]  Peter B. Shull,et al.  HaptiVec: Presenting Haptic Feedback Vectors in Handheld Controllers using Embedded Tactile Pin Arrays , 2019, CHI.

[20]  Jonathan Tong,et al.  Two-Point Orientation Discrimination Versus the Traditional Two-Point Test for Tactile Spatial Acuity Assessment , 2013, Front. Hum. Neurosci..

[21]  Alex Olwal,et al.  shiftIO: Reconfigurable Tactile Elements for Dynamic Affordances and Mobile Interaction , 2017, CHI.

[22]  Patrick Baudisch,et al.  Back-of-device interaction allows creating very small touch devices , 2009, CHI.

[23]  H. Kajimoto,et al.  HamsaTouch: tactile vision substitution with smartphone and electro-tactile display , 2014, CHI Extended Abstracts.

[24]  Sean Follmer,et al.  Applications of Switchable Permanent Magnetic Actuators in Shape Change and Tactile Display , 2016, UIST.

[25]  Gi-Hun Yang,et al.  T-mobile: Vibrotactile display pad with spatial and directional information for hand-held device , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Jeffrey P. Bigham,et al.  Making Everyday Interfaces Accessible: Tactile Overlays by and for Blind People , 2018, IEEE Pervasive Computing.

[27]  Ali Israr,et al.  Tactile brush: drawing on skin with a tactile grid display , 2011, CHI.

[28]  Jon Froehlich,et al.  TacTILE: A Preliminary Toolchain for Creating Accessible Graphics with 3D-Printed Overlays and Auditory Annotations , 2017, ASSETS.

[29]  Robert Kincaid Tactile guides for touch screen controls , 2012, BCS HCI.

[30]  Chris Harrison,et al.  Providing dynamically changeable physical buttons on a visual display , 2009, CHI.

[31]  Alan C. Brady,et al.  Results from a Tactile Array on the Fingertip , 2003 .

[32]  Ivan Poupyrev,et al.  Actuation and tangible user interfaces: the Vaucanson duck, robots, and shape displays , 2007, TEI.

[33]  Jan O. Borchers,et al.  HaptiCase: Back-of-Device Tactile Landmarks for Eyes-Free Absolute Indirect Touch , 2015, CHI.

[34]  Fabian Hemmert,et al.  Dynamic knobs: shape change as a means of interaction on a mobile phone , 2008, CHI Extended Abstracts.

[35]  F. Vidal-Verdu,et al.  Graphical Tactile Displays for Visually-Impaired People , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[36]  Khalil Najafi,et al.  Electrostatically driven micro-hydraulic actuator arrays , 2010, 2010 IEEE 23rd International Conference on Micro Electro Mechanical Systems (MEMS).

[37]  Ki-Suk Kim,et al.  Two-point discrimination values vary depending on test site, sex and test modality in the orofacial region: a preliminary study , 2017, Journal of applied oral science : revista FOB.

[38]  Velko Vechev,et al.  MagTics: Flexible and Thin Form Factor Magnetic Actuators for Dynamic and Wearable Haptic Feedback , 2017, UIST.

[39]  Koji Yatani,et al.  SemFeel: a user interface with semantic tactile feedback for mobile touch-screen devices , 2009, UIST '09.

[40]  Gerhard Weber,et al.  Tactile Graphics Revised: The Novel BrailleDis 9000 Pin-Matrix Device with Multitouch Input , 2008, ICCHP.

[41]  Fabian Hemmert,et al.  Shape-changing mobiles: tapering in one-dimensional deformational displays in mobile phones , 2010, TEI '10.

[42]  Meredith Ringel Morris,et al.  Touchplates: low-cost tactile overlays for visually impaired touch screen users , 2013, ASSETS.

[43]  Hyoukryeol Choi,et al.  Development of Soft-Actuator-Based Wearable Tactile Display , 2008, IEEE Transactions on Robotics.

[44]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[45]  Jennifer Pearson,et al.  Pulp Friction: Exploring the Finger Pad Periphery for Subtle Haptic Feedback , 2019, CHI.

[46]  Xiaojun Bi,et al.  RearType: text entry using keys on the back of a device , 2010, Mobile HCI.

[47]  L. Bickley,et al.  Comprar Bates Guide to Physical Examination and History Taking | Lynn S. Bickley MD | 9781605474007 | Lippincott Williams & Wilkins , 2008 .

[48]  Geehyuk Lee,et al.  Using Poke Stimuli to Improve a 3x3 Watch-back Tactile Display , 2019, MobileHCI.

[49]  Yuriko Nakai,et al.  Tactile Presentation to the Back of a Smartphone with Simultaneous Screen Operation , 2016, CHI Extended Abstracts.

[50]  Tonya L. Smith-Jackson,et al.  Touch-screens are not tangible: fusing tangible interaction with touch glass in readers for the blind , 2013, TEI '13.

[51]  Allison M. Okamura,et al.  WAVES: A Wearable Asymmetric Vibration Excitation System for Presenting Three-Dimensional Translation and Rotation Cues , 2017, CHI.