Reducing Latency by Eliminating Synchrony

Drawing or dragging an object on a mobile device is annoying today because the latency is manifested spatially with an obvious gap between the touch point and the line head or dragged object. This work identifies the multiple synchronization points in the input to display path of modern mobile systems as a major source of latency, contributing about 30 ms to the overall latency. We present Presto, an asynchronous design of the input to display path. By focusing on the main application and relaxing conventional requirements of no frame drop and no tearing effects, Presto is able to eliminate much of the latency due to synchrony. By carefully guarding against consecutive frame drops and limiting the risk of tearing to a small region around the touch point, Presto is able to reduce their visual impact to barely noticeable. Using a prototype based on Android 5, we are able to quantify the effectiveness, overhead and user experience of Presto through both objective measurements and subjective user assessment. We show that Presto is able to reduce the latency of legacy Android applications by close to half; and more importantly, we show this reduction is orthogonal to that by other popular approaches. When combined with touch prediction, Presto is able to reduce the touch latency below 10 ms, a remarkable achievement without any hardware support.

[1]  MyungJoo Ham,et al.  LPD: Low Power Display Mechanism for Mobile and Wearable Devices , 2015, USENIX Annual Technical Conference.

[2]  Zheng Wang,et al.  Using latency to evaluate interactive system performance , 1996, OSDI '96.

[3]  References , 1971 .

[4]  Edward Lank,et al.  Endpoint prediction using motion kinematics , 2007, CHI.

[5]  Lin Zhong,et al.  POLYPATH: Supporting Multiple Tradeoffs for Interaction Latency , 2016, ArXiv.

[6]  Mahadev Satyanarayanan,et al.  Towards wearable cognitive assistance , 2014, MobiSys.

[7]  Gerard O'Regan.  Texas Instruments , 1964, Nature.

[8]  Kajal T. Claypool,et al.  The effects of frame rate and resolution on users playing first person shooter games , 2006, Electronic Imaging.

[9]  Ehud Sharlin,et al.  Predictive interaction using the delphian desktop , 2005, UIST.

[10]  Kevin Forseth,et al.  Graphics for Architecture , 1979 .

[11]  Robert J. Teather,et al.  Is 60 FPS better than 30?: the impact of frame rate and latency on moving target selection , 2014, CHI Extended Abstracts.

[12]  Scott A. Mahlke,et al.  Accelerating Mobile Applications through Flip-Flop Replication , 2015, MobiSys.

[13]  D. W. Zimmerman.  Teacher’s Corner: A Note on Interpretation of the Paired-Samples t Test , 1997 .

[14]  Daniel J. Wigdor,et al.  Designing for low-latency direct-touch input , 2012, UIST.

[15]  Ben Shneiderman,et al.  Determining Causes and Severity of End-User Frustration , 2004, Int. J. Hum. Comput. Interact..

[16]  Joseph J. LaViola,et al.  Double exponential smoothing: an alternative to Kalman filter-based predictive tracking , 2003, IPT/EGVE.

[17]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[18]  Christina Freytag,et al.  Designing And Engineering Time The Psychology Of Time Perception In Software , 2016 .

[19]  Alec Wolman,et al.  Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming , 2015, MobiSys.

[20]  Michael B. Jones,et al.  CPU reservations and time constraints: efficient, predictable scheduling of independent activities , 1997, SOSP.

[21]  Xu Chen,et al.  COMET: Code Offload by Migrating Execution Transparently , 2012, OSDI.

[22]  Jacob O. Wobbrock,et al.  Mouse pointing endpoint prediction using kinematic template matching , 2014, CHI.

[23]  Anoop Gupta,et al.  In the blink of an eye: investigating latency perception during stylus interaction , 2014, CHI.

[24]  Emery D. Berger,et al.  Usenix Association 8th Usenix Symposium on Operating Systems Design and Implementation 73 Redline: First Class Support for Interactivity in Commodity Operating Systems , 2022 .

[25]  Kajal T. Claypool,et al.  On frame rate and player performance in first person shooter games , 2007, Multimedia Systems.

[26]  Jonathan Walpole,et al.  Supporting time-sensitive applications on a commodity OS , 2002, OPSR.

[27]  Margo I. Seltzer,et al.  Improving interactive performance using TIPME , 2000, SIGMETRICS '00.

[28]  Daniel J. Wigdor,et al.  How Much Faster is Fast Enough?: User Perception of Latency & Latency Improvements in Direct and Indirect Touch , 2015, CHI.

[29]  D. W. Zimmerman.  Teacher’s Corner: A Note on Interpretation of the Paired-Samples t Test , 1997 .