Internet Protocol (IP) Network Measurement, Characterization, Modeling, and Control for Self-Managed Networks

Abstract : IP network technology cannot continue on an ever-increasing course of technological complexity and yet require the kind of human intervention that is necessary today for network management. Networks must be self-managing. This can only be done by a system of measurement that copes with the dynamics of packet movement. This system must process the packet-level measurements into variables that characterize network behaviors, which then form the basis for control algorithms that react to the variables. Such packet level measurements can lead to characterization and control at the application layer, at the transport layer, at the network layer (including overlay networking), and in some cases at the link layer. The research documented in this report used tools of statistics, data mining and machine learning to (1) determine network variables derivable from the measurements that characterize network behavior; (2) develop models of the critical network variables that characterize performance, usage, security, and early onset of problems; and (3) develop automated control methods based on the variables.

[1]  Herwig Bruneel,et al.  Performance of Telecommunication Systems: Selected Topics , 2002 .

[2]  Edward W. Knightly,et al.  Second moment resource allocation in multi-service networks , 1997, SIGMETRICS '97.

[3]  C H Chang,et al.  EFFECTIVE BANDWIDTH IN HIGHSPEED DIGITAL NETWORKS , 1995 .

[4]  Hamid Ahmadi,et al.  Equivalent Capacity and Its Application to Bandwidth Allocation in High-Speed Networks , 1991, IEEE J. Sel. Areas Commun..

[5]  Murad S. Taqqu,et al.  On the Self-Similar Nature of Ethernet Traffic , 1993, SIGCOMM.

[6]  John W. Tukey,et al.  Data Analysis and Regression: A Second Course in Statistics , 1977 .

[7]  R. Weber,et al.  Buffer overflow asymptotics for a buffer handling many traffic sources , 1996, Journal of Applied Probability.

[8]  N. Duffield,et al.  Economies of scale in queues with sources having power-law large deviation scalings , 1996, Journal of Applied Probability.

[9]  Richard G. Baraniuk,et al.  A Multifractal Wavelet Model with Application to Network Traffic , 1999, IEEE Trans. Inf. Theory.

[10]  Izhak Rubin,et al.  Multiplicative multifractal modelling of long-range-dependent network traffic , 2001, Int. J. Commun. Syst..

[11]  Gustavo de Veciana,et al.  Stability and performance analysis of networks supporting elastic services , 2001, TNET.

[12]  Cheng-Shang Chang,et al.  On the performance of multiplexing independent regulated inputs , 2001, SIGMETRICS '01.

[13]  Debasis Mitra,et al.  A New Approach for Allocating Buffers and Bandwidth to Heterogeneous Regulated Traffic in an ATM Node , 1995, IEEE J. Sel. Areas Commun..

[14]  Ward Whitt,et al.  Effective bandwidths with priorities , 1998, TNET.

[15]  Ward Whitt,et al.  On the effectiveness of effective bandwidths for admission control in ATM networks , 1994 .

[16]  Debasis Mitra,et al.  Effective bandwidth of general Markovian traffic sources and admission control of high speed networks , 1993, IEEE INFOCOM '93 The Conference on Computer Communications, Proceedings.

[17]  Jin Cao,et al.  A Poisson limit for buffer overflow probabilities , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[18]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[19]  George D. Stamoulis,et al.  Application of the many sources asymptotic and effective bandwidths to traffic engineering , 1999, Telecommun. Syst..

[20]  Christophe Diot,et al.  An approach to alleviate link overload as observed on an IP backbone , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[21]  Alain Simonian,et al.  Large Deviations Approximation for Fluid Queues Fed by a Large Number of on/off Sources , 1994 .

[22]  Ciro D'Apice,et al.  Queueing Theory , 2003, Operations Research.

[23]  Fouad A. Tobagi,et al.  Provisioning IP backbone networks to support latency sensitive traffic , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[24]  Ilkka Norros,et al.  A storage model with self-similar input , 1994, Queueing Syst. Theory Appl..

[25]  I. Adan,et al.  QUEUEING THEORY , 1978 .

[26]  George C. Polyzos,et al.  A Parameterizable Methodology for Internet Traffic Flow Profiling , 1995, IEEE J. Sel. Areas Commun..

[27]  Michel Mandjes,et al.  The shape of the loss curve and the impact of long-range dependence on network performance , 2001 .

[28]  J. R. M. Hosking,et al.  FRACTIONAL DIFFERENCING MODELING IN HYDROLOGY , 1985 .

[29]  Richard A. Becker,et al.  The Visual Design and Control of Trellis Display , 1996 .

[30]  Nick G. Duffield,et al.  Large deviations, the shape of the loss curve, and economies of scale in large multiplexers , 1995, Queueing Syst. Theory Appl..

[31]  T. V. Lakshman,et al.  Fundamental Bounds and Approximations for ATM Multiplexers with Applications to Video Teleconferencing , 1995, IEEE J. Sel. Areas Commun..

[32]  George Kesidis,et al.  Worst-case performance of a buffer with independent shaped arrival processes , 2000, IEEE Communications Letters.

[33]  W. Richard Stevens,et al.  TCP/IP Illustrated, Volume 1: The Protocols , 1994 .

[34]  Jin Cao,et al.  On the nonstationarity of Internet traffic , 2001, SIGMETRICS '01.

[35]  Thomas Bonald,et al.  Statistical bandwidth sharing: a study of congestion at flow level , 2001, SIGCOMM.

[36]  Costas Courcoubetis,et al.  Procedures and tools for analysis of network traffic measurements , 2002, Perform. Evaluation.

[37]  Iraj Saniee,et al.  Performance impacts of multi-scaling in wide area TCP/IP traffic , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[38]  Michel Mandjes,et al.  Large Deviations for Small Buffers: An Insensitivity Result , 2001, Queueing Syst. Theory Appl..

[39]  Donald F. Towsley,et al.  Statistical Analysis of Generalized Processor Sharing Scheduling Discipline , 1995, IEEE J. Sel. Areas Commun..

[40]  Vern Paxson,et al.  Automated packet trace analysis of TCP implementations , 1997, SIGCOMM '97.

[41]  Richard G. Baraniuk,et al.  Simulation of nonGaussian long-range-dependent traffic using wavelets , 1999, SIGMETRICS '99.

[42]  Ravi Mazumdar,et al.  Cell loss asymptotics in buffers fed with a large number of independent stationary sources , 1998, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[43]  Jin Cao,et al.  Internet Traffic Tends Toward Poisson and Independent as the Load Increases , 2003 .

[44]  Walter Willinger,et al.  Experimental queueing analysis with long-range dependent packet traffic , 1996, TNET.

[45]  Joseph Y. Hui.  Resource allocation for broadband networks , 1988, IEEE J. Sel. Areas Commun..

[46]  Frank Kelly,et al.  Notes on effective bandwidths , 1994 .

[47]  Ness B. Shroff,et al.  Admission control for statistical QoS: theory and practice , 1999, IEEE Netw..

[48]  Jim Kurose,et al.  Statistical analysis of generalized processor sharing scheduling discipline , 1994, SIGCOMM 1994.

[49]  Anja Feldmann,et al.  Measurement and analysis of IP network usage and behavior , 2000, IEEE Commun. Mag..

[50]  M M Ky Yo O Oo O Bo.  FAST SIMULATION OF A QUEUE FED BY A SUPERPOSITION OF MANY (HEAVY-TAILED) SOURCES , 2002 .