Markovian Modeling of Real Data Traffic: Heuristic Phase Type and MAP Fitting of Heavy Tailed and Fractal Like Samples

In order to support the effective use of telecommunication infrastructure, the "random" behavior of traffic sources has been studied since the early days of telephony. Strange new features, like fractal like behavior and heavy tailed distributions were observed in high speed packet switched data networks in the early '90s. Since that time a fertile research aims to find proper models to describe these strange traffic features and to establish a robust method to design, dimension and operate such networks.In this paper we give an overview of methods that, on the one hand, allow us to capture important traffic properties like slow decay rate, Hurst parameter, scaling factor, etc., and, on the other hand, makes possible the quantitative analysis of the studied systems using the effective analysis approach called matrix geometric method.The presentation of this analysis approach is associated with a discussion on the properties and limits of Markovian fitting of the typical non-Markovian behavior present in telecommunication networks.

[1]  Ramin Sadre,et al.  Fitting World Wide Web request traces with the EM-algorithim , 2001, SPIE ITCom.

[2]  S. Lowen,et al.  Point process models for self-similar network traffic, with applications , 1998 .

[3]  T. Rydén.  Estimating the order of continuous phase-type distributions and Markov-modulated Poisson processes , 1997 .

[4]  Ramin Sadre,et al.  A validation of the pseudo self-similar traffic model , 2002, Proceedings International Conference on Dependable Systems and Networks.

[5]  Richard G. Baraniuk,et al.  A Multifractal Wavelet Model with Application to Network Traffic , 1999, IEEE Trans. Inf. Theory.

[6]  Jan Beran,et al.  Statistics for long-memory processes , 1994 .

[7]  Henry J. Fowler,et al.  Local Area Network Traffic Characteristics, with Implications for Broadband Network Congestion Management , 1991, IEEE J. Sel. Areas Commun..

[8]  Sidney I. Resnick,et al.  Heavy Tail Modelling and Teletraffic Data , 1995 .

[9]  B. M. Hill,et al.  A Simple General Approach to Inference About the Tail of a Distribution , 1975 .

[10]  Enrique Castillo.  Extreme value theory in engineering , 1988 .

[11]  M. Crovella,et al.  Estimating the Heavy Tail Index from Scaling Properties , 1999 .

[12]  Vaidyanathan Ramaswami,et al.  Introduction to Matrix Analytic Methods in Stochastic Modeling , 1999, ASA-SIAM Series on Statistics and Applied Mathematics.

[13]  Anja Feldmann,et al.  Fitting Mixtures of Exponentials to Long-Tail Distributions to Analyze Network , 1998, Perform. Evaluation.

[14]  Sem C. Borst,et al.  Heavy Tails: The Effect of the Service Discipline , 2002, Computer Performance Evaluation / TOOLS.

[15]  B. Mandelbrot,et al.  Fractional Brownian Motions, Fractional Noises and Applications , 1968 .

[16]  T. Rydén.  Parameter Estimation for Markov Modulated Poisson Processes , 1994 .

[17]  C. Granger,et al.  AN INTRODUCTION TO LONG‐MEMORY TIME SERIES MODELS AND FRACTIONAL DIFFERENCING , 1980 .

[18]  Ren Asmussen,et al.  Fitting Phase-type Distributions via the EM Algorithm , 1996 .

[19]  David M. Lucantoni,et al.  New results for the single server queue with a batch Markovian arrival process , 1991 .

[20]  Anja Feldmann,et al.  Data networks as cascades: investigating the multifractal nature of Internet WAN traffic , 1998, SIGCOMM '98.

[21]  T. Rydén.  An EM algorithm for estimation in Markov-modulated Poisson processes , 1996 .

[22]  Rudolf H. Riedi,et al.  An introduction to multifractals , 1997 .

[23]  J. L. Véhel,et al.  Fractional Brownian motion and data traffic modeling: The other end of the spectrum , 1997 .

[24]  Jean-Yves Le Boudec,et al.  New Models for Pseudo Self-Similar Traffic , 1997, Perform. Evaluation.

[25]  J. Rice.  Mathematical Statistics and Data Analysis , 1988 .

[26]  Bo Friis Nielsen,et al.  A Markovian approach for modeling packet traffic with long-range dependence , 1998, IEEE J. Sel. Areas Commun..

[27]  Walter Willinger,et al.  On the self-similar nature of Ethernet traffic , 1993, SIGCOMM '93.

[28]  S. Resnick,et al.  The qq-estimator and heavy tails , 1996 .

[29]  K. Meier-Hellstern.  A fitting algorithm for Markov-modulated poisson processes having two arrival rates , 1987 .

[30]  Ilkka Norros,et al.  On the Use of Fractional Brownian Motion in the Theory of Connectionless Networks , 1995, IEEE J. Sel. Areas Commun..

[31]  Miklós Telek,et al.  A MARKOVIAN POINT PROCESS EXHIBITING MULTIFRACTAL BEHAVIOR AND ITS APPLICATION TO TRAFFIC MODELING , 2002 .

[32]  Ilkka Norros,et al.  A storage model with self-similar input , 1994, Queueing Syst. Theory Appl..

[33]  Sidney Resnick,et al.  Smoothing the Hill Estimator , 1997, Advances in Applied Probability.

[34]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[35]  S. Resnick.  Heavy tail modeling and teletraffic data: special invited paper , 1997 .

[36]  M. Taqqu,et al.  Large-Sample Properties of Parameter Estimates for Strongly Dependent Stationary Gaussian Time Series , 1986 .

[37]  Marcel F. Neuts,et al.  Structured Stochastic Matrices of M/G/1 Type and Their Applications , 1989 .

[38]  Miklós Telek,et al.  Acyclic discrete phase type distributions: properties and a parameter estimation algorithm , 2003, Perform. Evaluation.

[39]  David M. Lucantoni,et al.  A Markov Modulated Characterization of Packetized Voice and Data Traffic and Related Statistical Multiplexer Performance , 1986, IEEE J. Sel. Areas Commun..

[40]  Ramin Sadre,et al.  Fitting World Wide Web request traces with the EM-algorithim , 2001, SPIE ITCom.

[41]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[42]  Marcel F. Neuts,et al.  Matrix-Geometric Solutions in Stochastic Models , 1981 .

[43]  Wolfgang Fischer,et al.  The Markov-Modulated Poisson Process (MMPP) Cookbook , 1993, Perform. Evaluation.

[44]  Miklós Telek,et al.  PhFit: a general phase-type fitting tool , 2002, Proceedings International Conference on Dependable Systems and Networks.

[45]  A. Cumani.  On the canonical representation of homogeneous markov processes modelling failure - time distributions , 1982 .