Reflection: Automated test location selection for cellular network upgrades

Cellular networks are constantly evolving due to frequent changes in radio access and end user equipment technologies, dynamic applications and associated trafflc mixes. Network upgrades should be performed with extreme caution since millions of users heavily depend on the cellular networks for a wide range of day to day tasks, including emergency and alert notifications. Before upgrading the entire network, it is important to conduct field evaluation of upgrades. Field evaluations are typically cumbersome and can be time consuming; however if done correctly they can help alleviate a lot of the deployment issues in terms of service quality degradation. The choice and number of field test locations have significant impacts on the time-to-market as well as confidence in how well various network upgrades will work out in the rest of the network. In this paper, we propose a novel approach — Reflection to automatically determine where to conduct the upgrade field tests in order to accurately identify important features that affect the upgrade. We demonstrate the effectiveness of Reflection using extensive evaluation based on real traces collected from a major US cellular network as well as synthetic traces.

[1]  Nick Feamster,et al.  Answering “What-If” Deployment and Configuration Questions With WISE: Techniques and Deployment Experience , 2008, IEEE/ACM Transactions on Networking.

[2]  Srinivasan Seshan,et al.  Developing a predictive model of quality of experience for internet video , 2013, SIGCOMM.

[3]  Andreas Haeberlen,et al.  The Good, the Bad, and the Differences: Better Network Diagnostics with Differential Provenance , 2016, SIGCOMM.

[4]  Michel Minoux,et al.  Accelerated greedy algorithms for maximizing submodular set functions , 1978 .

[5]  A. Jefferson Offutt,et al.  Combination testing strategies: a survey , 2005, Softw. Test. Verification Reliab..

[6]  N. K. Shankaranarayanan,et al.  Magus: minimizing cellular service disruption during network upgrades , 2015, CoNEXT.

[7]  Xin Jin,et al.  Dynamic scheduling of network updates , 2014, SIGCOMM.

[8]  Swarun Kumar,et al.  LTE radio analytics made easy and accessible , 2015, SIGCOMM 2015.

[9]  Daniel Massey,et al.  Argus: End-to-end service anomaly detection and localization from an ISP's point of view , 2012, 2012 Proceedings IEEE INFOCOM.

[10]  Mona Attariyan,et al.  X-ray: Automating Root-Cause Diagnosis of Performance Anomalies in Production Software , 2012, OSDI.

[11]  Gregory R. Ganger,et al.  Diagnosing Performance Changes by Comparing Request Flows , 2011, NSDI.

[12]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[13]  Rongrong Wang,et al.  Restricted Isometry Property of Random Subdictionaries , 2015, IEEE Transactions on Information Theory.

[14]  Jie Yang,et al.  Proactive call drop avoidance in UMTS networks , 2013, 2013 Proceedings IEEE INFOCOM.

[15]  Daniel Massey,et al.  G-RCA: a generic root cause analysis platform for service quality management in large IP networks , 2012, TNET.

[16]  Yin Zhang,et al.  Troubleshooting chronic conditions in large IP networks , 2008, CoNEXT '08.

[17]  Shobha Venkataraman,et al.  A first look at cellular network performance during crowded events , 2013, SIGMETRICS '13.

[18]  Matthew Caesar,et al.  Walk the line: consistent network updates with bandwidth guarantees , 2012, HotSDN '12.

[19]  Srikanth Kandula,et al.  Achieving high utilization with software-driven WAN , 2013, SIGCOMM.

[20]  Yin Zhang,et al.  Rapid detection of maintenance induced changes in service performance , 2011, CoNEXT '11.

[21]  Jing Xu,et al.  Robust assessment of changes in cellular networks , 2013, CoNEXT.

[22]  Paramvir Bahl,et al.  Detailed diagnosis in enterprise networks , 2009, SIGCOMM '09.

[23]  David Walker,et al.  Abstractions for network update , 2012, SIGCOMM '12.

[24]  Feng Qian,et al.  An in-depth study of LTE: effect of network protocol and application behavior on performance , 2013, SIGCOMM.

[25]  Fernando Silveira,et al.  URCA: Pulling out Anomalies by their Root Causes , 2010, 2010 Proceedings IEEE INFOCOM.

[26]  Songwu Lu,et al.  Control-plane protocol interactions in cellular networks , 2014, SIGCOMM.

[27]  Qi Zhao,et al.  Towards automated performance diagnosis in a large IPTV network , 2009, SIGCOMM '09.

[28]  Ron Kohavi,et al.  Online controlled experiments at large scale , 2013, KDD.

[29]  Ron Kohavi,et al.  Practical guide to controlled experiments on the web: listen to your customers not to the hippo , 2007, KDD '07.

[30]  Yin Zhang,et al.  Detecting the performance impact of upgrades in large operational networks , 2010, SIGCOMM 2010.

[31]  Xin Wu,et al.  zUpdate: updating data center networks with zero loss , 2013, SIGCOMM.

[32]  Randy H. Katz,et al.  An algebraic approach to practical and scalable overlay network monitoring , 2004, SIGCOMM 2004.