Graph-Based Deep Learning for Fast and Tight Network Calculus Analyses

Network Calculus computes delay bounds for individual data flows in networks of aggregate schedulers. It searches for the best model bounding resource contention between these flows at each scheduler. The literature proposes different analyses to consider realistic behavior of networked system such as multiplexing and contention between flows in consecutive queues. Moreover, not a single of the existing NC heuristics that are based on an algebraic analysis is strictly best. An exhaustive search for the best combination of analyses was proposed with the Tandem Matching Analysis (TMA). Additional measures made it scale best among the NC analyses, yet bounding delays may still require several hours of computation. In this paper, we demonstrate the ability to couple graph-based neural networks with NC by extending TMA with a prediction mechanism replacing the exhaustive search. We propose a framework that learns from NC's TMA, predicts best contention models and feeds them back to TMA where the according NC computations are executed. We achieve provably valid bounds that are very competitive with the exhaustive TMA. We observe a maximum relative error to TMA below 12%, while execution times remain nearly constant and outperform TMA in differently sized networks by several orders of magnitude.

[1]  Eric Thierry,et al.  Tight performance bounds in the worst-case analysis of feed-forward networks , 2010, 2010 Proceedings IEEE INFOCOM.

[2]  Fabien Geyer,et al.  DeepComNet: Performance evaluation of network topologies using graph-based deep learning , 2019, Perform. Evaluation.

[3]  Mor Harchol-Balter,et al.  PriorityMeister: Tail Latency QoS for Shared Networked Storage , 2014, SoCC.

[4]  Rene L. Cruz,et al.  A calculus for network delay, Part II: Network analysis , 1991, IEEE Trans. Inf. Theory.

[5]  Marc Boyer.  NC-Maude: A Rewriting Tool to Play with Network Calculus , 2010, ISoLA.

[6]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[7]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[8]  Bodo Rosenhahn,et al.  Neural Networks for Measurement-based Bandwidth Estimation , 2018, 2018 IFIP Networking Conference (IFIP Networking) and Workshops.

[9]  Justine Sherry,et al.  Silo: Predictable Message Latency in the Cloud , 2015, Comput. Commun. Rev..

[10]  Felix Poloczek,et al.  Sharp per-flow delay bounds for bursty arrivals: The case of FIFO, SP, and EDF scheduling , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[11]  Georg Carle,et al.  Network engineering for real-time networks: comparison of automotive and aeronautic industries approaches , 2016, IEEE Communications Magazine.

[12]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[13]  Jens B. Schmitt,et al.  Boosting sensor network calculus by thoroughly bounding cross-traffic , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[14]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[15]  Ivan Martinovic,et al.  Improving Performance Bounds in Feed-Forward Networks by Paying Multiplexing Only Once , 2008, MMB.

[16]  A. Cabellos-Aparicio,et al.  RouteNet: Leveraging Graph Neural Networks for Network Modeling and Optimization in SDN , 2019, IEEE Journal on Selected Areas in Communications.

[17]  Giovanni Stea,et al.  Exact Worst-Case Delay in FIFO-Multiplexing Feed-Forward Networks , 2015, IEEE/ACM Transactions on Networking.

[18]  Jean-Yves Le Boudec,et al.  Delay Bounds in a Network with Aggregate Scheduling , 2000, QofIS.

[19]  Marc Boyer,et al.  Embedding network calculus and event stream theory in a common model , 2016, 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA).

[20]  Jens B. Schmitt,et al.  Delay Bounds under Arbitrary Multiplexing: When Network Calculus Leaves You in the Lurch... , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[21]  Fabien Geyer,et al.  On the Robustness of Deep Learning-predicted Contention Models for Network Calculus , 2020, 2020 IEEE Symposium on Computers and Communications (ISCC).

[22]  David L. Dill,et al.  Learning a SAT Solver from Single-Bit Supervision , 2018, ICLR.

[23]  Cheng-Shang Chang,et al.  Performance guarantees in communication networks , 2000, Eur. Trans. Telecommun..

[24]  Ramin Yahyapour,et al.  An analytical model for software defined networking: A network calculus-based approach , 2013, 2013 IEEE Global Communications Conference (GLOBECOM).

[25]  Lothar Thiele,et al.  Analytic real-time analysis and timed automata: a hybrid method for analyzing embedded real-time systems , 2009, EMSOFT '09.

[26]  Fujun Wang,et al.  Survey on Learning-Based Formal Methods: Taxonomy, Applications and Possible Future Directions , 2020, IEEE Access.

[27]  Ryo Nakamura,et al.  On Estimating Communication Delays using Graph Convolutional Networks with Semi-Supervised Learning , 2020, 2020 International Conference on Information Networking (ICOIN).

[28]  Fabien Geyer,et al.  Performance Evaluation of Network Topologies using Graph-Based Deep Learning , 2017, VALUETOOLS.

[29]  Krzysztof Rusek,et al.  Message-Passing Neural Networks Learn Little’s Law , 2019, IEEE Communications Letters.

[30]  Fang Dong,et al.  Copula analysis for statistical network calculus , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[31]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[32]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[33]  Michael A. Beck,et al.  Towards a statistical network calculus — Dealing with uncertainty in arrivals , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[34]  Rene L. Cruz,et al.  A calculus for network delay, Part I: Network elements in isolation , 1991, IEEE Trans. Inf. Theory.

[35]  Wolfgang Kellerer,et al.  DetServ: Network Models for Real-Time QoS Provisioning in SDN-Based Industrial Environments , 2017, IEEE Transactions on Network and Service Management.

[36]  Luís C. Lamb,et al.  Learning to Solve NP-Complete Problems - A Graph Neural Network for the Decision TSP , 2018, AAAI.

[37]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[38]  Fabien Geyer,et al.  DeepTMA: Predicting Effective Contention Models for Network Calculus using Graph Neural Networks , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[39]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[40]  Jens B. Schmitt,et al.  Quality and Cost of Deterministic Network Calculus: Design and Evaluation of an Accurate and Fast Analysis , 2016, SIGMETRICS.

[41]  Jens B. Schmitt,et al.  Achieving Efficiency without Sacrificing Model Accuracy: Network Calculus on Compact Domains , 2016, 2016 IEEE 24th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS).

[42]  Jianping Wang,et al.  Learning-Based Dynamic Resource Provisioning for Network Slicing with Ensured End-to-End Performance Bound , 2020, IEEE Transactions on Network Science and Engineering.

[43]  Mark G. Karpovsky,et al.  Application of network calculus to general topologies using turn-prohibition , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[44]  Almut Burchard,et al.  A (min, ×) network calculus for multi-hop fading channels , 2013, 2013 Proceedings IEEE INFOCOM.

[45]  Alexander Scheffler,et al.  The Deterministic Network Calculus Analysis: Reliability Insights and Performance Improvements , 2018, 2018 IEEE 23rd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD).

[46]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[47]  Florin Ciucu,et al.  A network service curve approach for the stochastic analysis of networks , 2005, SIGMETRICS '05.

[48]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[49]  James F. Kurose,et al.  A network calculus for cache networks , 2013, 2013 Proceedings IEEE INFOCOM.

[50]  Lothar Thiele,et al.  Real-time calculus for scheduling hard real-time systems , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[51]  Wang Yi,et al.  Generalized finitary real-time calculus , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[52]  Cynthia Rudin,et al.  All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously , 2019, J. Mach. Learn. Res..

[53]  Jens B. Schmitt,et al.  The DiscoDNC v2 - A Comprehensive Tool for Deterministic Network Calculus , 2014, VALUETOOLS.

[54]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[55]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[56]  Cynthia Rudin,et al.  Model Class Reliance: Variable Importance Measures for any Machine Learning Model Class, from the "Rashomon" Perspective , 2018 .

[57]  Jens B. Schmitt,et al.  Calculating Accurate End-to-End Delay Bounds - You Better Know Your Cross-Traffic , 2016, EAI Endorsed Trans. Ubiquitous Environ..

[58]  Jörg Liebeherr.  Duality of the Max-Plus and Min-Plus Network Calculus , 2017, Found. Trends Netw..

[59]  Moussa Amrani,et al.  ML + FV = $\heartsuit$? A Survey on the Application of Machine Learning to Formal Verification , 2018, 1806.03600.

[60]  Sebastian Vastag,et al.  Modeling quantitative requirements in SLAs with network calculus , 2011, VALUETOOLS.

[61]  Georg Carle,et al.  The Case for a Network Calculus Heuristic: Using Insights from Data for Tighter Bounds , 2018, 2018 30th International Teletraffic Congress (ITC 30).

[62]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[63]  Jintao Wang,et al.  Analyzing Multimode Wireless Sensor Networks Using the Network Calculus , 2015, J. Sensors.

[64]  Wang Yi,et al.  Finitary Real-Time Calculus: Efficient Performance Analysis of Distributed Embedded Systems , 2013, 2013 IEEE 34th Real-Time Systems Symposium.