POLYPATH: Supporting Multiple Tradeoffs for Interaction Latency

Modern mobile systems use a single input-to-display path to serve all applications. In meeting the visual goals of all applications, the path has a latency inadequate for many important interactions. To accommodate the different latency requirements and visual constraints by different interactions, we present POLYPATH, a system design in which application developers (and users) can choose from multiple path designs for their application at any time. Because a POLYPATH system asks for two or more path designs, we present a novel fast path design, called Presto. Presto reduces latency by judiciously allowing frame drops and tearing. We report an Android 5-based prototype of POLYPATH with two path designs: Android legacy and Presto. Using this prototype, we quantify the effectiveness, overhead, and user experience of POLYPATH, especially Presto, through both objective measurements and subjective user assessment. We show that Presto reduces the latency of legacy touchscreen drawing applications by almost half; and more importantly, this reduction is orthogonal to that of other popular approaches and is achieved without any user-noticeable negative visual effect. When combined with touch prediction, Presto is able to reduce the touch latency below 10 ms, a remarkable achievement without any hardware support.

[1]  Eddie Kohler,et al.  The Click modular router , 1999, SOSP.

[2]  Alec Wolman,et al.  Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming , 2015, MobiSys.

[3]  Robert J. Teather,et al.  Is 60 FPS better than 30?: the impact of frame rate and latency on moving target selection , 2014, CHI Extended Abstracts.

[4]  Robert B. Miller,et al.  Response time in man-computer conversational transactions , 1899, AFIPS Fall Joint Computing Conference.

[5]  Mark Segal,et al.  The OpenGL Graphics System: A Specification , 2004 .

[6]  Joseph J. LaViola,et al.  Double exponential smoothing: an alternative to Kalman filter-based predictive tracking , 2003, IPT/EGVE.

[7]  Scott A. Mahlke,et al.  Accelerating Mobile Applications through Flip-Flop Replication , 2015, MobiSys.

[8]  Brian N. Bershad,et al.  Extensibility safety and performance in the SPIN operating system , 1995, SOSP.

[9]  Christina Freytag,et al.  Designing And Engineering Time The Psychology Of Time Perception In Software , 2016 .

[10]  Yiran Chen,et al.  Mobile GPU Power Consumption Reduction via Dynamic Resolution and Frame Rate Scaling , 2014, HotPower.

[11]  Yu Yan,et al.  Optimizing power consumption of mobile games , 2015, HotPower '15.

[12]  Steven M. LaValle,et al.  Head tracking for the Oculus Rift , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Jonathan Walpole,et al.  Supporting time-sensitive applications on a commodity OS , 2002, OPSR.

[14]  Erwan Petillon Demystify DSI I/F , 2012 .

[15]  Margo I. Seltzer,et al.  Improving interactive performance using TIPME , 2000, SIGMETRICS '00.

[16]  Daniel J. Wigdor,et al.  How Much Faster is Fast Enough?: User Perception of Latency & Latency Improvements in Direct and Indirect Touch , 2015, CHI.

[17]  MyungJoo Ham,et al.  LPD: Low Power Display Mechanism for Mobile and Wearable Devices , 2015, USENIX Annual Technical Conference.

[18]  Gerard O'Regan Texas Instruments , 1964, Nature.

[19]  Ehud Sharlin,et al.  Predictive interaction using the delphian desktop , 2005, UIST.

[20]  Larry L. Peterson,et al.  Making paths explicit in the Scout operating system , 1996, OSDI '96.

[21]  Edward Lank,et al.  Endpoint prediction using motion kinematics , 2007, CHI.

[22]  Zheng Wang,et al.  Using latency to evaluate interactive system performance , 1996, OSDI '96.

[23]  Michael B. Jones,et al.  CPU reservations and time constraints: efficient, predictable scheduling of independent activities , 1997, SOSP.

[24]  Xu Chen,et al.  COMET: Code Offload by Migrating Execution Transparently , 2012, OSDI.

[25]  Anoop Gupta,et al.  In the blink of an eye: investigating latency perception during stylus interaction , 2014, CHI.

[26]  Emery D. Berger,et al.  Usenix Association 8th Usenix Symposium on Operating Systems Design and Implementation 73 Redline: First Class Support for Interactivity in Commodity Operating Systems , 2022 .

[27]  Kajal T. Claypool,et al.  On frame rate and player performance in first person shooter games , 2007, Multimedia Systems.

[28]  Peter Henderson,et al.  A lazy evaluator , 1976, POPL.

[29]  Kajal T. Claypool,et al.  Latency and player actions in online games , 2006, CACM.

[30]  Jacob O. Wobbrock,et al.  Mouse pointing endpoint prediction using kinematic template matching , 2014, CHI.

[31]  Larry L. Peterson,et al.  The x-Kernel: An Architecture for Implementing Network Protocols , 1991, IEEE Trans. Software Eng..

[32]  Jay Lepreau,et al.  The Flux OSKit: a substrate for kernel and language research , 1997, SOSP.

[33]  Julia L. Lawall,et al.  Proceedings of the 2002 Usenix Annual Technical Conference Think: a Software Framework for Component-based Operating System Kernels , 2022 .

[34]  Feng Zhao,et al.  Rethinking Energy-Performance Trade-Off in Mobile Web Page Loading , 2015, GETMBL.

[35]  Kajal T. Claypool,et al.  The effects of frame rate and resolution on users playing first person shooter games , 2006, Electronic Imaging.

[36]  Kajal T. Claypool,et al.  Latency can kill: precision and deadline in online games , 2010, MMSys '10.

[37]  Niraj K. Jha,et al.  Towards a responsive, yet power-efficient, operating system: a holistic approach , 2005, 13th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems.

[38]  Steven C. Seow Designing and Engineering Time: The Psychology of Time Perception in Software , 2008 .

[39]  Daniel J. Wigdor,et al.  Designing for low-latency direct-touch input , 2012, UIST.