Back-of-Device Force Feedback Improves Touchscreen Interaction for Mobile Devices

Touchscreen interaction suffers from occlusion problems as fingers can cover small targets, which makes interacting with such targets challenging. To improve touchscreen interaction accuracy and consequently the selection of small or hidden objects, we introduce a back-of-device force feedback system for smartphones. We introduce a new solution that combines force feedback on the back to enhance touch input on the front screen. The interface includes three actuated pins at the back of a smartphone. All three pins are driven by microservos and can be actuated up to a frequency of 50 Hz and a maximum amplitude of 5 mm. In a first psychophysical user study, we explored the limits of the system. Thereafter, we demonstrate through a performance study that the proposed interface can enhance touchscreen interaction precision, compared to state-of-the-art methods. In particular, the selection of small targets performed remarkably well with force feedback. The study additionally shows that users subjectively felt significantly more accurate with force feedback. Based on the results, we discuss back-to-front feedback design issues and demonstrate potential applications through several prototypical concepts to illustrate where the back-of-device force feedback could be beneficial.

[1]  Hiroshi Ishii,et al.  Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices , 2012, UIST.

[2]  Hai-Ning Liang,et al.  User-defined surface+motion gestures for 3d manipulation of objects at a distance through a mobile device , 2012, APCHI '12.

[3]  Darren Leigh,et al.  Under the table interaction , 2006, UIST.

[4]  Pattie Maes,et al.  Flexpad: highly flexible bending interactions for projected handheld displays , 2013, CHI.

[5]  Ali Israr,et al.  FeelSleeve: Haptic Feedback to Enhance Early Reading , 2015, CHI.

[6]  Motoyuki Akamatsu,et al.  A multi-modal mouse with tactile and force feedback , 1994, Int. J. Hum. Comput. Stud..

[7]  Caroline Jay,et al.  Delayed visual and haptic feedback in a reciprocal tapping task , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[8]  Pedro Lopes,et al.  Muscle-propelled force feedback: bringing force feedback to mobile devices , 2013, CHI.

[9]  Xiang 'Anthony' Chen,et al.  The fat thumb: using the thumb's contact size for single-handed mobile interaction , 2012, Mobile HCI.

[10]  Anne Marie Piper,et al.  Surface haptic interactions with a TPad tablet , 2013, UIST.

[11]  Takeshi Naemura,et al.  LimpiDual touch: interactive limpid display with dual-sided touch sensing , 2008, International Conference on Computer Graphics and Interactive Techniques.

[12]  Joaquim A. Jorge,et al.  Exploring the Non-Visual Acquisition of Targets on Touch Phones and Tablets , 2011 .

[13]  Jie Liu,et al.  Understanding performance of eyes-free, absolute position control on touchable mobile phones , 2013, MobileHCI '13.

[14]  Xiang Xiao,et al.  LensGesture: augmenting mobile interactions with back-of-device finger gestures , 2013, ICMI '13.

[15]  Jan O. Borchers,et al.  BackXPress: Using Back-of-Device Finger Pressure to Augment Touchscreen Input on Smartphones , 2017, CHI.

[16]  Antti Oulasvirta,et al.  Sandwich keyboard: fast ten-finger typing on a mobile device with adaptive touch sensing on the back side , 2013, MobileHCI '13.

[17]  M. Sheelagh T. Carpendale,et al.  The Haptic Tabletop Puck: tactile feedback for interactive tabletops , 2009, ITS '09.

[18]  Roel Vertegaal,et al.  Unifone: designing for auxiliary finger input in one-handed mobile interactions , 2013, TEI '13.

[19]  Pourang Irani,et al.  Thumbs-Up: 3D Spatial Thumb-Reachable Space for One-Handed Thumb Interaction on Smartphones , 2016, SUI.

[20]  Seung-Chan Kim,et al.  Tactile rendering of 3D features on touch surfaces , 2013, UIST.

[21]  Richard E. Ladner,et al.  Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities , 2009, Assets '09.

[22]  Benjamin B. Bederson,et al.  Target size study for one-handed thumb use on small touchscreen devices , 2006, Mobile HCI.

[23]  Hiroshi Ishii,et al.  Haptic Edge Display for Mobile Tactile Interaction , 2016, CHI.

[24]  Karon E. MacLean,et al.  Evaluation of haptically augmented touchscreen gui elements under cognitive load , 2007, ICMI '07.

[25]  Ivan Poupyrev,et al.  Drawing with feeling: designing tactile display for pen , 2002, SIGGRAPH '02.

[26]  Michael Rohs,et al.  Ergonomic characteristics of gestures for front- and back-of-tablets interaction with grasping hands , 2014, MobileHCI '14.

[27]  Patrick Baudisch,et al.  Lucid touch: a see-through mobile device , 2007, UIST.

[28]  M Akamatsu,et al.  Please Scroll down for Article Ergonomics a Comparison of Tactile, Auditory, and Visual Feedback in a Pointing Task Using a Mouse-type Device , 2022 .

[29]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[30]  W.J. Tompkins,et al.  Electrotactile and vibrotactile displays for sensory substitution systems , 1991, IEEE Transactions on Biomedical Engineering.

[31]  Jan O. Borchers,et al.  HaptiCase: Back-of-Device Tactile Landmarks for Eyes-Free Absolute Indirect Touch , 2015, CHI.

[32]  Buntarou Shizuki,et al.  A study of touch accuracy using smartphone cases with a dimple or a wedge shaped attachment , 2014 .

[33]  Lorna M. Brown,et al.  Tactile feedback for mobile interactions , 2007, CHI.

[34]  Martin Pielot,et al.  PocketMenu: non-visual menus for touch screen devices , 2012, Mobile HCI.

[35]  Brad A. Myers,et al.  The performance of hand postures in front- and back-of-device interaction for mobile computing , 2008, Int. J. Hum. Comput. Stud..

[36]  George Ghinea,et al.  Multilayer Haptic Feedback for Pen-Based Tablet Interaction , 2019, CHI.

[37]  Yuriko Nakai,et al.  Tactile Presentation to the Back of a Smartphone with Simultaneous Screen Operation , 2016, CHI Extended Abstracts.

[38]  Per Ola Kristensson,et al.  Improving two-thumb text entry on touchscreen devices , 2013, CHI.

[39]  Darren Leigh,et al.  Hammer Time!: A Low-Cost, High Precision, High Accuracy Tool to Measure the Latency of Touchscreen Devices , 2016, CHI.

[40]  Eve E. Hoggan,et al.  Playing with Tactile Feedback Latency in Touchscreen Interaction: Two Approaches , 2011, INTERACT.

[41]  Niels Henze,et al.  Fingers' Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen , 2018, CHI.

[42]  Stephen A. Brewster,et al.  Towards the Temporally Perfect Virtual Button: Touch-Feedback Simultaneity and Perceived Quality in Mobile Touchscreen Press Interactions , 2014, TAP.

[43]  Patrick Baudisch,et al.  Back-of-device interaction allows creating very small touch devices , 2009, CHI.

[44]  Chris Harrison,et al.  Texture displays: a passive approach to tactile presentation , 2009, CHI.

[45]  Jan O. Borchers,et al.  MudPad: tactile feedback and haptic texture overlay for touch surfaces , 2010, ITS '10.

[46]  Masaaki Fukumoto.  PuyoSheet and PuyoDots: simple techniques for adding "button-push" feeling to touch panels , 2009, CHI Extended Abstracts.

[47]  Mike Sinclair,et al.  TouchMover: actuated 3D touchscreen with haptic feedback , 2013, ITS.

[48]  I. Scott MacKenzie,et al.  A Model of Two-Thumb Text Entry , 2002, Graphics Interface.

[49]  Toshiaki Sugimura,et al.  Active click: tactile feedback for touch panels , 2001, CHI Extended Abstracts.

[50]  Masanori Sugimoto,et al.  HybridTouch: an intuitive manipulation technique for PDAs using their front and rear surfaces , 2006, Mobile HCI.

[51]  Robert Kincaid.  Tactile guides for touch screen controls , 2012, BCS HCI.

[52]  George Ghinea,et al.  ForceTab: Visuo-haptic interaction with a force-sensitive actuated tablet , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).