Proactive Serving Decreases User Delay Exponentially

In online service systems, the delay experienced by a user from the service request to the service completion is one of the most critical performance metrics. To improve user delay experience, recent industrial practice suggests a modern system design mechanism: proactive serving, where the system predicts future user requests and allocates its capacity to serve these upcoming requests proactively. In this paper, we investigate the fundamentals of proactive serving from a theoretical perspective. In particular, we show that proactive serving decreases average delay exponentially (as a function of the prediction window size). Our results provide theoretical foundations for proactive serving and shed light on its application in practical systems.

[1]  Carla Schlatter Ellis,et al.  Practical prefetching techniques for parallel file systems , 1991, [1991] Proceedings of the First International Conference on Parallel and Distributed Information Systems.

[2]  Cheng-Hsin Hsu,et al.  SmartTransfer: transferring your mobile multimedia contents at the "right" time , 2012, NOSSDAV '12.

[3]  Ye Xu,et al.  Preference, context and communities: a multi-faceted approach to predicting smartphone app usage patterns , 2013, ISWC '13.

[4]  Ibrahim Matta,et al.  Describing and forecasting video access patterns , 2011, 2011 Proceedings IEEE INFOCOM.

[5]  Stathes Hadjiefthymiades,et al.  Predicting the location of mobile users: a machine learning approach , 2009, ICPS '09.

[6]  James R. Larus,et al.  Branch prediction for free , 1993, PLDI '93.

[7]  Vincenzo Grassi,et al.  Modeling and evaluation of prefetching policies for context-aware information services , 1998, MobiCom '98.

[8]  Joseph Bak,et al.  The Residue Theorem , 2010 .

[9]  Asuman E. Ozdaglar,et al.  On the Delay and Throughput Gains of Coding in Unreliable Networks , 2008, IEEE Transactions on Information Theory.

[10]  Vincent S. Tseng,et al.  Efficient mining and prediction of user behavior patterns in mobile web systems , 2006, Inf. Softw. Technol..

[11]  Stanley B. Zdonik,et al.  Fido: A Cache That Learns to Fetch , 1991, VLDB.

[12]  Jeffrey C. Mogul,et al.  Using predictive prefetching to improve World Wide Web latency , 1996, CCRV.

[13]  Dvir Shabtay,et al.  Scheduling unit length jobs on parallel machines with lookahead information , 2011, J. Sched..

[14]  Minghua Chen,et al.  Effect of proactive serving on user delay reduction in service systems , 2014, SIGMETRICS '14.

[15]  Richard W. Vuduc,et al.  When Prefetching Works, When It Doesn’t, and Why , 2012, TACO.

[16]  Atilla Eryilmaz,et al.  Proactive Data Download and User Demand Shaping for Data Networks , 2013, ArXiv.

[17]  D. Zats,et al.  DeTail: reducing the flow completion time tail in datacenter networks , 2012, CCRV.

[18]  Jason Flinn,et al.  Informed mobile prefetching , 2012, MobiSys '12.

[19]  S. Wittevrongel,et al.  Queueing Systems , 2019, Introduction to Stochastic Processes and Simulation.

[20]  Mark Handley,et al.  The resource pooling principle , 2008, CCRV.

[21]  Seung Jun Baek,et al.  Reducing delays by network coding for wireless broadcasting in networks using relay stations , 2012, 2012 IEEE 23rd International Symposium on Personal, Indoor and Mobile Radio Communications - (PIMRC).

[22]  G. Veciana,et al.  Throughput optimality of delay-driven MaxWeight scheduler for a wireless system with flow dynamics , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  Michael J. Neely.  Delay-Based Network Utility Maximization , 2013, IEEE/ACM Transactions on Networking.

[24]  Ness B. Shroff,et al.  Delay-based Back-Pressure scheduling in multi-hop wireless networks , 2011, INFOCOM.

[25]  B. Chandrasekaran.  Survey of Network Traffic Models , 2006 .

[26]  Joel H. Spencer,et al.  Queueing with future information , 2014, PERV.

[27]  Xin Chen,et al.  Coordinated data prefetching for web contents , 2005, Comput. Commun..

[28]  Brian D. Noble,et al.  BreadCrumbs: forecasting mobile connectivity , 2008, MobiCom '08.

[29]  Haifeng Yu,et al.  DRAM-page based prediction and prefetching , 2000, Proceedings 2000 International Conference on Computer Design.

[30]  Matthew Andrews,et al.  Providing quality of service over a shared wireless link , 2001, IEEE Commun. Mag..

[31]  Rex K. Kincaid,et al.  A look-ahead heuristic for scheduling jobs with release dates on a single machine , 1994, Comput. Oper. Res..

[32]  Xifeng Yan,et al.  Workload characterization and prediction in the cloud: A multiple time series approach , 2012, 2012 IEEE Network Operations and Management Symposium.

[33]  Brian D. Noble,et al.  Bobtail: Avoiding Long Tails in the Cloud , 2013, NSDI.

[34]  Arnold O. Allen,et al.  Probability, statistics and queueing theory - with computer science applications (2. ed.) , 1981, Int. CMG Conference.

[35]  Xin Chen,et al.  A Popularity-Based Prediction Model for Web Prefetching , 2003, Computer.

[36]  Stephan Sigg,et al.  Development of a novel context prediction algorithm and analysis of context prediction schemes , 2008 .

[37]  Murray Hill,et al.  SCHEDULING IN A QUEUING SYSTEM WITH ASYNCHRONOUSLY VARYING SERVICE RATES , 2004 .

[38]  Minghua Chen,et al.  When Backpressure Meets Predictive Scheduling , 2013, IEEE/ACM Transactions on Networking.

[39]  G. J. A. Stern,et al.  Queueing Systems, Volume 2: Computer Applications , 1976 .

[40]  Terri Watson,et al.  Application Design for Wireless Computing , 1994, 1994 First Workshop on Mobile Computing Systems and Applications.

[41]  Hai Jin,et al.  A Measurement Study of a Peer-to-Peer Video-on-Demand System , 2007, IPTPS.

[42]  Milica Stojanovic,et al.  On Coding for Delay—Network Coding for Time-Division Duplexing , 2012, IEEE Transactions on Information Theory.

[43]  Ron Kohavi,et al.  Online Experiments: Lessons Learned , 2007, Computer.

[44]  Ben Coleman.  Quality vs. Performance in Lookahead Scheduling , 2006, JCIS.

[45]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[46]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[47]  Klara Nahrstedt,et al.  Peer-to-peer multimedia streaming and caching service , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[48]  M. Medard,et al.  On Delay Performance Gains From Network Coding , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[49]  Jim Zelenka,et al.  Informed prefetching and caching , 1995, SOSP.

[50]  Alois Ferscha,et al.  Recognizing and Predicting Context by Learning from User Behavior 1 , 2003 .

[51]  Madhu Sudan,et al.  Queuing with future information , 2012 .

[52]  Kannan Ramchandran,et al.  Codes can reduce queueing delay in data centers , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[53]  Atilla Eryilmaz,et al.  Proactive Resource Allocation: Harnessing the Diversity and Multicast Gains , 2011, IEEE Transactions on Information Theory.

[54]  Lizy Kurian John,et al.  Store-Load-Branch (SLB) predictor: A compiler assisted branch prediction for data dependent branches , 2013, 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA).

[55]  Alexander L. Stolyar,et al.  Scheduling for multiple flows sharing a time-varying channel: the exponential rule , 2000 .