PalmTouch: Using the Palm as an Additional Input Modality on Commodity Smartphones

Touchscreens are the most successful input method for smartphones. Despite their flexibility, touch input is limited to the location of taps and gestures. We present PalmTouch, an additional input modality that differentiates between touches of fingers and the palm. Touching the display with the palm can be a natural gesture since moving the thumb towards the device's top edge implicitly places the palm on the touchscreen. We present different use cases for PalmTouch, including the use as a shortcut and for improving reachability. To evaluate these use cases, we have developed a model that differentiates between finger and palm touch with an accuracy of 99.53% in realistic scenarios. Results of the evaluation show that participants perceive the input modality as intuitive and natural to perform. Moreover, they appreciate PalmTouch as an easy and fast solution to address the reachability issue during one-handed smartphone interaction compared to thumb stretching or grip changes.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  Simon Rogers,et al.  AnglePose: robust, precise capacitive touch tracking via 3d orientation estimation , 2011, CHI.

[3]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[4]  Roel Vertegaal,et al.  Unifone: designing for auxiliary finger input in one-handed mobile interactions , 2013, TEI '13.

[5]  Robert Xiao,et al.  CapAuth: Identifying and Differentiating User Handprints on Commodity Capacitive Touchscreens , 2015, ITS.

[6]  Eric Lecolinet,et al.  MicroRolls: expanding touch-screen input vocabulary by distinguishing rolls vs. slides of the thumb , 2009, CHI.

[7]  Robert Xiao,et al.  Estimating 3D Finger Angle on Commodity Touchscreens , 2015, ITS.

[8]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Bing-Yu Chen,et al.  GaussSense: attachable stylus sensing using magnetic sensor grid , 2012, UIST '12.

[10]  Jonna Häkkilä,et al.  Exploring finger specific touch screen interaction for mobile phone user interfaces , 2014, OZCHI.

[11]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[12]  Ian Oakley,et al.  The Flat Finger: Exploring Area Touches on Smartwatches , 2016, CHI.

[13]  Marc Langheinrich,et al.  Back-of-device authentication on smartphones , 2013, CHI.

[14]  Rowel O. Atienza,et al.  Implicit Palm Rejection Using Real-Time Hand Model Filters on Tablet Devices , 2015, 2015 9th International Conference on Next Generation Mobile Applications, Services and Technologies.

[15]  Chris Harrison,et al.  TapSense: enhancing finger interaction on touch surfaces , 2011, UIST.

[16]  Xiang 'Anthony' Chen,et al.  The fat thumb: using the thumb's contact size for single-handed mobile interaction , 2012, Mobile HCI.

[17]  Wonyong Sung,et al.  Structured Pruning of Deep Convolutional Neural Networks , 2015, ACM J. Emerg. Technol. Comput. Syst..

[18]  Patrick Schrempf,et al.  RadarCat: Radar Categorization for Input & Interaction , 2016, UIST.

[19]  Geehyuk Lee,et al.  Forcetap: extending the input vocabulary of mobile touch screens by adding tap gestures , 2011, Mobile HCI.

[20]  Antonio Krüger,et al.  Same-side Hand Interactions with Arm-placed Devices Using EMG , 2015, CHI Extended Abstracts.

[21]  Jun Rekimoto,et al.  GraspZoom: zooming and scrolling control model for single-handed mobile interaction , 2009, Mobile HCI.

[22]  Joanna Bergstrom-Lehtovirta,et al.  Modeling the functional area of the thumb on mobile touchscreen surfaces , 2014, CHI.

[23]  J·施瓦茨,et al.  Probabilistic palm rejection using spatiotemporal touch features and iterative classification , 2015 .

[24]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[25]  Otmar Hilliges,et al.  In-air gestures around unmodified mobile devices , 2014, UIST.

[26]  Senaka Buthpitiya,et al.  Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts , 2015, CHI.

[27]  Jane Yung-jen Hsu,et al.  Double-side multi-touch input for mobile devices , 2009, CHI Extended Abstracts.

[28]  Martin Halvey,et al.  Investigating one-handed multi-digit pressure input for mobile devices , 2012, CHI EA '12.

[29]  Rich Caruana,et al.  Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping , 2000, NIPS.

[30]  Niels Henze,et al.  Interaction Methods and Use Cases for a Full-Touch Sensing Smartphone , 2017, CHI Extended Abstracts.

[31]  Sarah Morrison-Smith,et al.  Exploring User-Defined Back-Of-Device Gestures for Mobile Devices , 2015, MobileHCI.

[32]  Per Ola Kristensson,et al.  Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices , 2017, CHI.

[33]  Patrick Olivier,et al.  Expressy: Using a Wrist-worn Inertial Measurement Unit to Add Expressiveness to Touch-based Interactions , 2016, CHI.

[34]  Niels Henze,et al.  Estimating the Finger Orientation on Capacitive Touchscreens Using Convolutional Neural Networks , 2017, ISS.

[35]  Chris Harrison,et al.  Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction , 2012, CHI.

[36]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[37]  Niels Henze,et al.  Investigating Screen Shifting Techniques to Improve One-Handed Smartphone Usage , 2016, NordiCHI.

[38]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[39]  Niels Henze,et al.  Design and evaluation of a layered handheld 3d display with touch-sensitive front and back , 2014, NordiCHI.

[40]  Niels Henze,et al.  A smartphone prototype for touch interaction on the whole device surface , 2017, MobileHCI.

[41]  Niels Henze,et al.  Fingers' Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen , 2018, CHI.

[42]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[43]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[44]  Patrick Baudisch,et al.  Back-of-device interaction allows creating very small touch devices , 2009, CHI.

[45]  Ian Oakley,et al.  TriTap: Identifying Finger Touches on Smartwatches , 2017, CHI.

[46]  S. Lederman,et al.  Human Hand Function , 2006 .

[47]  Matt J Camilleri,et al.  Touch displays: the effects of palm rejection technology on productivity, comfort, biomechanics and positioning , 2013, Ergonomics.

[48]  Geehyuk Lee,et al.  LongPad: a touchpad using the entire area below the keyboard of a laptop computer , 2013, CHI.

[49]  Ashley Colley,et al.  Identifying unintentional touches on handheld touch screen devices , 2012, DIS '12.

[50]  J. B. Brooke,et al.  SUS: a retrospective , 2013 .