Improving software-reduced touchscreen latency

The latency of current mobile devices' touchscreens is around 100ms and has widely been explored. Latency down to 2ms is noticeable, and latency as low as 25ms reduces users' performance. Previous work reduced touch latency by extrapolating a finger's movement using an ensemble of shallow neural networks and showed that predicting 33ms into the future increases users' performance. Unfortunately, this prediction has a high error. Predicting beyond 33ms did not increase participants' performance, and the error affected the subjective assessment. We use more recent machine learning techniques to reduce the prediction error. We train LSTM networks and multilayer perceptrons using a large data set and regularization. We show that linear extrapolation causes an 116.7% higher error and the previously proposed ensembles of shallow networks cause a 26.7% higher error compared to the LSTM networks. The trained models, the data used for testing, and the source code is available on GitHub.

[1]  Yvette Hackett Does Practice Make Perfect? , 2003 .

[2]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[5]  Anoop Gupta,et al.  In the blink of an eye: investigating latency perception during stylus interaction , 2014, CHI.

[6]  Daniel J. Wigdor,et al.  Designing for low-latency direct-touch input , 2012, UIST.

[7]  Markus Funk,et al.  Software-reduced touchscreen latency , 2016, MobileHCI.

[8]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[9]  Daniel J. Wigdor,et al.  How fast is fast enough?: a study of the effects of latency in direct-touch pointing tasks , 2013, CHI.

[10]  Niels Henze,et al.  App stores: external validity for mobile HCI , 2013, INTR.

[11]  Renaud Blanch,et al.  Two touch system latency estimators: high accuracy and low overhead , 2013, ITS.

[12]  Pascal Perrier,et al.  Reducing Latency with a Continuous Prediction: Effects on Users' Performance in Direct-Touch Target Acquisitions , 2015, ITS.

[13]  Alireza Sahami Shirazi,et al.  Empirical Research through Ubiquitous Data Collection , 2013, Computer.

[14]  Darren Leigh,et al.  Hammer Time!: A Low-Cost, High Precision, High Accuracy Tool to Measure the Latency of Touchscreen Devices , 2016, CHI.

[15]  Stephen A. Brewster,et al.  Feedback is... late: measuring multimodal delays in mobile device touchscreen interaction , 2010, ICMI-MLMI '10.

[16]  Glen J. Anderson,et al.  User Perception of Touch Screen Latency , 2011, HCI.

[17]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.