A user-specific machine learning approach for improving touch accuracy on mobile devices

We present a flexible Machine Learning approach for learning user-specific touch input models to increase touch accuracy on mobile devices. The model is based on flexible, non-parametric Gaussian Process regression and is learned using recorded touch inputs. We demonstrate that significant touch accuracy improvements can be obtained when either raw sensor data is used as an input or when the device's reported touch location is used as an input, with the latter marginally outperforming the former. We show that learned offset functions are highly nonlinear and user-specific and that user-specific models outperform models trained on data pooled from several users. Crucially, significant performance improvements can be obtained with a small (≈200) number of training examples, easily obtained for a particular user through a calibration game or from keyboard entry data.

[1]  John Williamson,et al.  Continuous uncertain interaction , 2006 .

[2]  Scott E. Hudson,et al.  A framework for robust and flexible handling of inputs with uncertainty , 2010, UIST.

[3]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[4]  Niels Henze,et al.  100,000,000 taps: analysis and improvement of touch performance in the large , 2011, Mobile HCI.

[5]  Patrick Baudisch,et al.  Back-of-device interaction allows creating very small touch devices , 2009, CHI.

[6]  Patrick Baudisch,et al.  The generalized perceived input point model and how to double touch accuracy by extracting fingerprints , 2010, CHI.

[7]  Chris Harrison,et al.  PocketTouch: through-fabric capacitive touch input , 2011, UIST '11.

[8]  Simon Rogers,et al.  FingerCloud: uncertainty and autonomy handover incapacitive sensing , 2010, CHI.

[9]  Kenneth C. Smith,et al.  A multi-touch three dimensional touch-sensitive tablet , 1985, CHI '85.

[10]  David R. Flatla,et al.  Calibration games: making calibration tasks enjoyable by adding motivating game elements , 2011, UIST.

[11]  Benjamin B. Bederson,et al.  Target size study for one-handed thumb use on small touchscreen devices , 2006, Mobile HCI.

[12]  Niels Henze,et al.  Observational and experimental investigation of typing behaviour using virtual keyboards for mobile devices , 2012, CHI.

[13]  Simon Rogers,et al.  A First Course in Machine Learning , 2011, Chapman and Hall / CRC machine learning and pattern recognition series.

[14]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[15]  Tyler Baldwin,et al.  Towards online adaptation and personalization of key-target resizing for mobile devices , 2012, IUI '12.

[16]  Jacob O. Wobbrock,et al.  Personalized input: improving ten-finger touchscreen typing through automatic adaptation , 2012, CHI.

[17]  Simon Rogers,et al.  AnglePose: robust, precise capacitive touch tracking via 3d orientation estimation , 2011, CHI.

[18]  Adam J. Aviv,et al.  Smudge Attacks on Smartphone Touch Screens , 2010, WOOT.

[19]  Patrick Baudisch,et al.  Understanding touch , 2011, CHI.

[20]  Scott E. Hudson,et al.  Monte carlo methods for managing interactive state, action and feedback under uncertainty , 2011, UIST '11.