Accurate and Fast Recovery of Network Monitoring Data With GPU-Accelerated Tensor Completion

Monitoring the performance of a large network would involve a high measurement cost. To reduce the overhead, sparse network monitoring techniques may be applied to select paths or time intervals to take the measurements, while the remaining monitoring data can be inferred leveraging the spatial-temporal correlations among data. The quality of missing data recovery, however, highly relies on the specific inference technique adopted. Tensor completion is a promising technique for more accurate missing data inference by exploiting the multi-dimensional data structure. However, data processing for higher dimensional tensors involves a large amount of computation, which prevents conventional tensor completion algorithms from practical application in the presence of large amount of data. This work takes the initiative to investigate the potential and methodologies of performing parallel processing for high-speed and high accuracy tensor completion over Graphics Processing Units (GPUs). We propose a GPU-accelerated parallel Tensor Completion scheme (GPU-TC) for accurate and fast recovery of missing data. To improve the data recovery accuracy and speed, we propose three novel techniques to well exploit the tensor factorization structure and the GPU features: grid-based tensor partition, independent task assignment based on Fisher-Yates shuffle, sphere facilitated and memory-correlated scheduling. We have conducted extensive experiments using network traffic trace data to compare the proposed GPU-TC with the state of art tensor completion algorithms and matrix-based algorithms. The experimental results demonstrate that GPU-TC can achieve significantly better performance in terms of two relative error ratio metrics and computation time.

[1]  Brett W. Bader,et al.  The TOPHITS Model for Higher-Order Web Link Analysis∗ , 2006 .

[2]  Shimin Chen,et al.  Exploiting SIMD for complex numerical predicates , 2016, ICDE Workshops.

[3]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[4]  Peter J. Haas,et al.  Large-scale matrix factorization with distributed stochastic gradient descent , 2011, KDD.

[5]  Paul Barford,et al.  A signal analysis of network traffic anomalies , 2002, IMW '02.

[6]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Dafang Zhang,et al.  Accurate Recovery of Missing Network Measurement Data With Localized Tensor Completion , 2019, IEEE/ACM Transactions on Networking.

[8]  Yin Zhang,et al.  Robust network compressive sensing , 2014, MobiCom.

[9]  Jiannong Cao,et al.  Accurate Recovery of Internet Traffic Data: A Sequential Tensor Completion Approach , 2018, IEEE/ACM Transactions on Networking.

[10]  De LathauwerLieven Blind Separation of Exponential Polynomials and the Decomposition of a Tensor in Rank-$(L_r,L_r,1)$ Terms , 2011 .

[11]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[12]  Xin Wang,et al.  Graph based Tensor Recovery for Accurate Internet Anomaly Detection , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[13]  Gaogang Xie,et al.  Accurate recovery of Internet traffic data: A tensor completion approach , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[14]  Carter Bays,et al.  Improving a Poor Random Number Generator , 1976, TOMS.

[15]  Carsten Lund,et al.  Estimating point-to-point and point-to-multipoint traffic matrices: an information-theoretic approach , 2005, IEEE/ACM Transactions on Networking.

[16]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[17]  Konstantina Papagiannaki,et al.  Structural analysis of network traffic flows , 2004, SIGMETRICS '04/Performance '04.

[18]  J. Wishart Statistical tables , 2018, Global Education Monitoring Report.

[19]  Christos Faloutsos,et al.  FlexiFaCT: Scalable Flexible Factorization of Coupled Tensors on Hadoop , 2014, SDM.

[20]  Inderjit S. Dhillon,et al.  NOMAD: Nonlocking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion , 2013, Proc. VLDB Endow..

[21]  Hua Chen,et al.  Pingmesh: A Large-Scale System for Data Center Network Latency Measurement and Analysis , 2015, SIGCOMM.

[22]  Gaogang Xie,et al.  Accurate recovery of internet traffic data under dynamic measurements , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[23]  Guangdong Feng,et al.  A Tensor Based Method for Missing Traffic Data Completion , 2013 .

[24]  Chih-Jen Lin,et al.  A fast parallel SGD for matrix factorization in shared memory systems , 2013, RecSys.

[25]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[26]  Mark Crovella,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM '04.

[27]  Mark Crovella,et al.  On traffic matrix completion in the internet , 2012, Internet Measurement Conference.

[28]  Lei Shi,et al.  Dcell: a scalable and fault-tolerant network structure for data centers , 2008, SIGCOMM '08.

[29]  Matthew Roughan,et al.  Spatiotemporal Traffic Matrix Synthesis , 2015, Comput. Commun. Rev..

[30]  Jiannong Cao,et al.  Accurate Recovery of Internet Traffic Data Under Variable Rate Measurements , 2018, IEEE/ACM Transactions on Networking.

[31]  Walter Willinger,et al.  Spatio-Temporal Compressive Sensing and Internet Traffic Matrices (Extended Version) , 2012, IEEE/ACM Transactions on Networking.

[32]  Daniel M. Dunlavy,et al.  A scalable optimization approach for fitting canonical tensor decompositions , 2011 .

[33]  Peter D. Jarvis,et al.  Tensor Rank, Invariants, Inequalities, and Applications , 2012, SIAM J. Matrix Anal. Appl..

[34]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[35]  Jiannong Cao,et al.  Fast Tensor Factorization for Accurate Internet Anomaly Detection , 2017, IEEE/ACM Transactions on Networking.

[36]  Xiaoqian Jiang,et al.  Fast and Robust Parallel SGD Matrix Factorization , 2015, KDD.

[37]  Jiannong Cao,et al.  On-Line Anomaly Detection With High Accuracy , 2018, IEEE/ACM Transactions on Networking.

[38]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[39]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[40]  R. Nowak,et al.  Compressed Sensing for Networked Data , 2008, IEEE Signal Processing Magazine.

[41]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[42]  Guangdong Feng,et al.  Traffic volume data outlier recovery via tensor model , 2013 .

[43]  J. H. Choi,et al.  DFacTo: Distributed Factorization of Tensors , 2014, NIPS.

[44]  Gaogang Xie,et al.  Sequential and adaptive sampling for matrix completion in network monitoring systems , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[45]  Steve Uhlig,et al.  Providing public intradomain traffic matrices to the research community , 2006, CCRV.

[46]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[47]  Christos Faloutsos,et al.  GigaTensor: scaling tensor analysis up by 100 times - algorithms and discoveries , 2012, KDD.

[48]  Tamara G. Kolda,et al.  Categories and Subject Descriptors: G.4 [Mathematics of Computing]: Mathematical Software— , 2022 .

[49]  Rong Du,et al.  VANET based traffic estimation: A matrix completion approach , 2013, 2013 IEEE Global Communications Conference (GLOBECOM).

[50]  Zongpeng Li,et al.  deTector: a Topology-aware Monitoring System for Data Center Networks , 2017, USENIX Annual Technical Conference.

[51]  Nikos D. Sidiropoulos,et al.  SPLATT: Efficient and Parallel Sparse Tensor-Matrix Multiplication , 2015, 2015 IEEE International Parallel and Distributed Processing Symposium.

[52]  William M. Kantor,et al.  The mathematics of perfect shuffles , 1983 .

[53]  E. Davidson,et al.  Strategies for analyzing data from video fluorometric monitoring of liquid chromatographic effluents , 1981 .

[54]  Yun Liang,et al.  CuMF_SGD: Fast and Scalable Matrix Factorization , 2016, ArXiv.

[55]  C. Berenstein,et al.  Network tomography , 2007 .

[56]  Ítalo S. Cunha,et al.  Predicting and tracking internet path changes , 2011, SIGCOMM.

[57]  George Karypis,et al.  An Exploration of Optimization Algorithms for High Performance Tensor Completion , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[58]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[59]  Bin Ran,et al.  A New Traffic Prediction Method based on Dynamic Tensor Completion , 2013 .

[60]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[61]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[62]  Walter Willinger,et al.  Spatio-temporal compressive sensing and internet traffic matrices , 2009, SIGCOMM '09.

[63]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[65]  Morteza Mardani,et al.  Robust network traffic estimation via sparsity and low rank , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[66]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[67]  Lieven De Lathauwer,et al.  Blind Separation of Exponential Polynomials and the Decomposition of a Tensor in Rank-(Lr, Lr, 1) Terms , 2011, SIAM J. Matrix Anal. Appl..