Data-driven vibrotactile rendering of digital buttons on touchscreens

Abstract Interaction with physical buttons is an essential part of our daily routine. We use buttons daily to turn lights on, to call an elevator, to ring a doorbell, or even to turn on our mobile devices. Buttons have distinct response characteristics and are easily activated by touch. However, there is limited tactile feedback available for their digital counterparts displayed on touchscreens. Although mobile phones incorporate low-cost vibration motors to enhance touch-based interactions, it is not possible to generate complex tactile effects on touchscreens. It is also difficult to relate the limited vibrotactile feedback generated by these motors to different types of physical buttons. In this study, we focus on creating vibrotactile feedback on a touchscreen that simulates the feeling of physical buttons using piezo actuators attached to it. We first recorded and analyzed the force, acceleration, and voltage data from twelve participants interacting with three different physical buttons: latch, toggle, and push buttons. Then, a button-specific vibrotactile stimulus was generated for each button based on the recorded data. Finally, we conducted a three-alternative forced choice (3AFC) experiment with twenty participants to explore whether the resultant stimulus is distinct and realistic. In our experiment, participants were able to match the three digital buttons with their physical counterparts with a success rate of 83%. In addition, we harvested seven adjective pairs from the participants expressing their perceptual feeling of pressing the physical buttons. All twenty participants rated the degree of their subjective feelings associated with each adjective for all the physical and digital buttons investigated in this study. Our statistical analysis showed that there exist at least three adjective pairs for which participants have rated two out of three digital buttons similar to their physical counterparts.

[1]  John Hardy,et al.  Characterising the Physicality of Everyday Buttons , 2014, ITS '14.

[2]  Darren Edge,et al.  Haptic keyclick feedback improves typing speed and reduces typing errors on a flat keyboard , 2015, 2015 IEEE World Haptics Conference (WHC).

[3]  Stephen A. Brewster,et al.  Investigating the effectiveness of tactile feedback for mobile touchscreens , 2008, CHI.

[4]  Shana Smith,et al.  A haptic keypad design with a novel interactive haptic feedback method , 2015 .

[5]  Cagatay Basdogan,et al.  Psychophysical Evaluation of Change in Friction on an Ultrasonically-Actuated Touchscreen , 2018, IEEE Transactions on Haptics.

[6]  Xiang Cao,et al.  A Masking Study of Key-Click Feedback Signals on a Virtual Keyboard , 2012, EuroHaptics.

[7]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[8]  Hong Z. Tan,et al.  Design and Evaluation of Identifiable Key-Click Signals for Mobile Devices , 2011, IEEE Transactions on Haptics.

[9]  Roope Raisamo,et al.  Comparison of three designs for haptic button edges on touchscreens , 2010, 2010 IEEE Haptics Symposium.

[10]  Geehyuk Lee,et al.  Haptic feedback design for a virtual button along force-displacement curves , 2013, UIST.

[11]  Jocelyn Monnoyer,et al.  Ultrasonic Friction Modulation While Pressing Induces a Tactile Feedback , 2016, EuroHaptics.

[12]  Hong Z. Tan,et al.  A study of touch typing performance with keyclick feedback , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[13]  Topi Kaaresoja,et al.  Feel-good touch: finding the most pleasant tactile feedback for a mobile touch screen button , 2008, ICMI '08.

[14]  Pradeep K. Khosla,et al.  Psychophysical Characterization and Testbed Validation of a Wearable Vibrotactile Glove for Telemanipulation , 2003, Presence: Teleoperators & Virtual Environments.

[15]  Topi Kaaresoja.  Latency guidelines for touchscreen virtual button feedback , 2016 .

[16]  Quan Liu,et al.  Perceptual dimensionality of manual key clicks , 2018, 2018 IEEE Haptics Symposium (HAPTICS).

[17]  Sharif Razzaque,et al.  Tactile virtual buttons for mobile devices , 2003, CHI Extended Abstracts.

[18]  Cagatay Basdogan,et al.  Tactile perception of change in friction on an ultrasonically actuated glass surface , 2017, 2017 IEEE World Haptics Conference (WHC).

[19]  M. Masen,et al.  A review of fingerpad contact mechanics and friction and how this affects tactile perception , 2015 .

[20]  Bruce Banter,et al.  Touch Screens and Touch Surfaces are Enriched by Haptic Force‐Feedback , 2010 .

[21]  Gilbert Cockton,et al.  CHI '03 Extended Abstracts on Human Factors in Computing Systems , 2003, CHI 2003.

[22]  Stephen A. Brewster,et al.  Towards the Temporally Perfect Virtual Button: Touch-Feedback Simultaneity and Perceived Quality in Mobile Touchscreen Press Interactions , 2014, TAP.

[23]  Cagatay Basdogan,et al.  HapTable: An Interactive Tabletop Providing Online Haptic Feedback for Touch Gestures , 2019, IEEE Transactions on Visualization and Computer Graphics.

[24]  Gholamreza Ilkhani,et al.  Data-Driven Texture Rendering on an Electrostatic Tactile Display , 2017, Int. J. Hum. Comput. Interact..

[25]  Takashi Maeno,et al.  Realization of button click feeling by use of ultrasonic vibration and force feedback , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[26]  Roope Raisamo,et al.  Designing tactile feedback for piezo buttons , 2011, CHI.

[27]  Glenn Gamst,et al.  Applied Multivariate Research: Design and Interpretation , 2005 .

[28]  Christian Müller-Tomfelde.  Tabletops - Horizontal Interactive Displays , 2010, Human-Computer Interaction Series.

[29]  Philippe Lefèvre,et al.  Dynamics of fingertip contact during the onset of tangential slip , 2014, Journal of The Royal Society Interface.

[30]  Seungmoon Choi,et al.  Tactile effect design and evaluation for virtual buttons on a mobile device touchscreen , 2011, Mobile HCI.

[31]  Stephen A. Brewster,et al.  Crossmodal congruence: the look, feel and sound of touchscreen widgets , 2008, ICMI '08.

[32]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[33]  Joseph M. Romano,et al.  Creating Realistic Virtual Textures from Contact Acceleration Data , 2012, IEEE Transactions on Haptics.

[34]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[35]  Allison M. Okamura,et al.  Reality-based models for vibration feedback in virtual environments , 2001 .

[36]  Heather Culbertson,et al.  Modeling and Rendering Realistic Textures from Unconstrained Tool-Surface Interactions , 2014, IEEE Transactions on Haptics.

[37]  Geehyuk Lee,et al.  Effect of Tactile Feedback for Button GUI on Mobile Touch Devices , 2014 .

[38]  Cagatay Basdogan,et al.  Tactile Masking by Electrovibration , 2018, IEEE Transactions on Haptics.

[39]  Harald Reiterer,et al.  Tactile feedback enhanced hand gesture interaction at large, high-resolution displays , 2009, J. Vis. Lang. Comput..

[40]  Lynette A. Jones,et al.  Tactile Displays: Guidance for Their Design and Application , 2008, Hum. Factors.

[41]  Matthias Harders,et al.  User-based evaluation of data-driven haptic rendering , 2010, TAP.

[42]  J. Barbera,et al.  Contact mechanics , 1999 .

[43]  Heather Culbertson,et al.  One hundred data-driven haptic texture models and open-source methods for rendering on 3D objects , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[44]  Cagatay Basdogan,et al.  Roughness perception of virtual textures displayed by electrovibration on touch screens , 2017, 2017 IEEE World Haptics Conference (WHC).

[45]  Lorna M. Brown,et al.  Tactile feedback for mobile interactions , 2007, CHI.

[46]  John M. Hollerbach,et al.  Reality-Based Haptic Force Models of Buttons and Switches , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[47]  van J. Kuilenburg,et al.  A mechanistic approach to tactile friction , 2013 .

[48]  Shumin Zhai,et al.  The performance of touch screen soft buttons , 2009, CHI.

[49]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.