A Compressive Sensing-Based Approach to End-to-End Network Traffic Reconstruction

Estimation of end-to-end network traffic plays an important role in traffic engineering and network planning. The direct measurement of a network's traffic matrix consumes large amounts of network resources and is thus impractical in most cases. How to accurately construct traffic matrix remains a great challenge. This paper studies end-to-end network traffic reconstruction in large-scale networks. Applying compressive sensing theory, we propose a novel reconstruction method for end-to-end traffic flows. First, the direct measurement of partial Origin-Destination (OD) flows is determined by random measurement matrix, providing partial measurements. Then, we use the K-SVD approach to obtain a sparse matrix. Combined with compressive sensing, this partially known OD flow matrix can be used to recover the entire end-to-end network traffic matrix. Simulation results show that the proposed method can reconstruct end-to-end network traffic with a high degree of accuracy. Moreover, in comparison with previous methods, our approach exhibits a significant performance improvement.

[1]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[2]  Oey Endra.  Joint optimization of measurement matrix and sparse dictionary in compressive sensing , 2012, 2012 International Conference on Computer and Communication Engineering (ICCCE).

[3]  Konstantina Papagiannaki,et al.  Structural analysis of network traffic flows , 2004, SIGMETRICS '04/Performance '04.

[4]  Ke Xu,et al.  A Model Approach to the Estimation of Peer-to-Peer Traffic Matrices , 2014, IEEE Transactions on Parallel and Distributed Systems.

[5]  H. S. Shahhoseini,et al.  Intelligent IP traffic matrix estimation by neural network and genetic algorithm , 2011, 2011 IEEE 7th International Symposium on Intelligent Signal Processing.

[6]  Victor Lopez,et al.  Virtual network topology adaptability based on data analytics for traffic prediction , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[7]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[8]  Pierre Vandergheynst,et al.  Compressed Sensing and Redundant Dictionaries , 2007, IEEE Transactions on Information Theory.

[9]  Michael Zink,et al.  CECT: computationally efficient congestion-avoidance and traffic engineering in software-defined cloud data centers , 2018, Cluster Computing.

[10]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[11]  Zhihan Lv,et al.  A Joint Multi-Criteria Utility-Based Network Selection Approach for Vehicle-to-Infrastructure Networking , 2018, IEEE Transactions on Intelligent Transportation Systems.

[12]  Matthew Roughan,et al.  Spatiotemporal Traffic Matrix Synthesis , 2015, Comput. Commun. Rev..

[13]  Murali S. Kodialam,et al.  Traffic engineering in software defined networks , 2013, 2013 Proceedings IEEE INFOCOM.

[14]  Gaogang Xie,et al.  Accurate recovery of internet traffic data under dynamic measurements , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[15]  Fadi Al-Turjman,et al.  Information-centric framework for the Internet of Things (IoT): Traffic modeling & optimization , 2018, Future Gener. Comput. Syst..

[16]  Attahiru Sule Alfa,et al.  IP Traffic Matrix Estimation Methods: Comparisons and Improvements , 2006, 2006 IEEE International Conference on Communications.

[17]  Albert G. Greenberg,et al.  Fast accurate computation of large-scale IP traffic matrices from link loads , 2003, SIGMETRICS '03.

[18]  Morteza Mardani,et al.  Estimating Traffic and Anomaly Maps via Network Tomography , 2014, IEEE/ACM Transactions on Networking.

[19]  Tomonori Takeda,et al.  Traffic matrix estimation in large-scale IP networks , 2010, 2010 17th IEEE Workshop on Local & Metropolitan Area Networks (LANMAN).

[20]  Thierry Chonavel,et al.  Efficient methods for traffic matrix modeling and on-line estimation in large-scale IP networks , 2009, 2009 21st International Teletraffic Congress.

[21]  Konstantina Papagiannaki,et al.  A distributed approach to measure IP traffic matrices , 2004, IMC '04.

[22]  Tianguang Chu,et al.  Structure Regularized Traffic Monitoring for Traffic Matrix Estimation and Anomaly Detection by Link-Load Measurements , 2016, IEEE Transactions on Instrumentation and Measurement.

[23]  Liu Bingjie,et al.  The Minimax Method of Design of Measurement Matrices for Compressed Sensing Based on Incoherence Criterion , 2012, 2012 Fifth International Conference on Intelligent Computation Technology and Automation.

[24]  Carsten Lund,et al.  Estimating point-to-point and point-to-multipoint traffic matrices: an information-theoretic approach , 2005, IEEE/ACM Transactions on Networking.

[25]  Eiji Oki,et al.  Gradually Reconfiguring Virtual Network Topologies Based on Estimated Traffic Matrices , 2007, IEEE/ACM Transactions on Networking.

[26]  Attahiru Sule Alfa,et al.  Hybrid Techniques for Large-Scale IP Traffic Matrix Estimation , 2010, 2010 IEEE International Conference on Communications.

[27]  Jing-Jing Zhou,et al.  Multi-scale traffic matrix estimation algorithm , 2010, 2010 Second Pacific-Asia Conference on Circuits, Communications and System.

[28]  Dingde Jiang,et al.  An energy-efficient cooperative multicast routing in multi-hop wireless networks for smart medical applications , 2017, Neurocomputing.

[29]  Chen-Nee Chuah,et al.  Distributed measurement-aware routing: Striking a balance between measurement and traffic engineering , 2012, 2012 Proceedings IEEE INFOCOM.

[30]  Ke Liu,et al.  Combining partial direct measurements with an information-theoretic approach to estimate traffic matrix , 2010, 2010 3rd IEEE International Conference on Broadband Network and Multimedia Technology (IC-BNMT).

[31]  Peng Zhang,et al.  Energy-Efficient Multi-Constraint Routing Algorithm With Load Balancing for Smart City Applications , 2016, IEEE Internet of Things Journal.

[32]  Dingde Jiang,et al.  Fine-granularity inference and estimations to network traffic for SDN , 2018, PloS one.

[33]  Lili Yan,et al.  Research of Traffic Matrix Estimation in IP Networks , 2011, 2011 International Conference on Internet Technology and Applications.

[34]  Kim-Kwang Raymond Choo,et al.  A multi-objective software defined network traffic measurement , 2017 .

[35]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[36]  Hongke Zhang,et al.  An approach for efficient, accurate, and timely estimation of traffic matrices , 2014, 2014 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[37]  Chen Wei-wei,et al.  Measurement Matrixes in Compressed Sensing Theory , 2012 .

[38]  Myungjin Lee,et al.  Opportunistic Flow-Level Latency Estimation Using Consistent NetFlow , 2012, IEEE/ACM Transactions on Networking.

[39]  Klara Nahrstedt,et al.  DistTM: Collaborative traffic matrix estimation in distributed SDN control planes , 2016, 2016 IFIP Networking Conference (IFIP Networking) and Workshops.

[40]  Jun Luo,et al.  Efficient traffic matrix estimation for data center networks , 2013, 2013 IFIP Networking Conference.

[41]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[42]  Walter Willinger,et al.  Spatio-temporal compressive sensing and internet traffic matrices , 2009, SIGCOMM '09.

[43]  Christophe Diot,et al.  Design of IGP link weight changes for estimation of traffic matrices , 2004, IEEE INFOCOM 2004.

[44]  Stefano Salsano,et al.  Joint Energy Efficient and QoS-Aware Path Allocation and VNF Placement for Service Function Chaining , 2017, IEEE Transactions on Network and Service Management.

[45]  Masoud Sabaei,et al.  A scalable traffic engineering technique in an SDN‐based data center network , 2018, Trans. Emerg. Telecommun. Technol..

[46]  Zou Yang,et al.  A GA approach for traffic matrix estimation , 2009, 2009 2nd IEEE International Conference on Broadband Network & Multimedia Technology.

[47]  Sheng Wang,et al.  Towards accurate online traffic matrix estimation in software-defined networks , 2015, SOSR.