An EM-Algorithm for MAP Fitting from Real Traffic Data

For model based analysis of computer and telecommunication systems an appropriate representation of arrival and service processes is very important. Especially representations that can be used in analytical or numerical solution approaches like phase type (PH) distributions or Markovian arrival processes (MAPs) are useful. This paper presents an algorithm to fit the parameters of a MAP according to measured data. The proposed algorithm is of the expectation-maximization (EM-) type and extends known approaches for the parameter fitting of PH-distributions and hidden Markov chains. It is shown that the algorithm generates MAPs which approximate traces very well and especially capture the autocorrelation in the trace. Furthermore the approach can be combined with other more efficient but less accurate fitting techniques by computing initial MAPs with those techniques and improving these MAPs with the approach presented in this paper.

[1]  Alma Riska,et al.  An EM-based technique for approximating long-tailed data sets with PH distributions , 2004, Perform. Evaluation.

[2]  Mark S. Squillante,et al.  MATRIX-ANALYTIC ANALYSIS OF A MAP/PH/1 QUEUE FITTED TO WEB SERVER DATA , 2002 .

[3]  Anja Feldmann,et al.  Fitting Mixtures of Exponentials to Long-Tail Distributions to Analyze Network , 1998, Perform. Evaluation.

[4]  Peter G. Taylor,et al.  Matrix-analytic methods: theory and applications , 2002 .

[5]  Donald F. Towsley,et al.  Measurement and modelling of the temporal dependence in packet loss , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[6]  Christoph Lindemann,et al.  Modeling IP traffic using the batch Markovian arrival process , 2003, Perform. Evaluation.

[7]  Ren Asmussen,et al.  Fitting Phase-type Distributions via the EM Algorithm , 1996 .

[8]  Srinivas R. Chakravarthy,et al.  Matrix-Analytic Methods in Stochastic Models , 1996 .

[9]  Donald F. Towsley,et al.  Continuous-time hidden Markov models for network performance evaluation , 2002, Perform. Evaluation.

[10]  Alma Riska,et al.  Efficient fitting of long-tailed data sets into phase-type distributions , 2002, PERV.

[11]  T. Rydén.  Parameter Estimation for Markov Modulated Poisson Processes , 1994 .

[12]  Marcel F. Neuts,et al.  Algorithmic probability - a collection of problems , 1995, Stochastic modeling series.

[13]  Walter Willinger,et al.  On the self-similar nature of Ethernet traffic , 1993, SIGCOMM '93.

[14]  Peter W. Glynn,et al.  Computing Poisson probabilities , 1988, CACM.

[15]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[16]  Kavé Salamatian,et al.  Hidden Markov modeling for network communication channels , 2001, SIGMETRICS '01.

[17]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[18]  Wolfgang Fischer,et al.  The Markov-Modulated Poisson Process (MMPP) Cookbook , 1993, Perform. Evaluation.

[19]  Miklós Telek,et al.  Markovian Modeling of Real Data Traffic: Heuristic Phase Type and MAP Fitting of Heavy Tailed and Fractal Like Samples , 2002, Performance.

[20]  Ramin Sadre,et al.  Fitting World Wide Web request traces with the EM-algorithim , 2001, SPIE ITCom.

[21]  Yutaka Takahashi,et al.  Practical Time-Scale Fitting of Self-Similar Traffic with Markov-Modulated Poisson Process , 2001, Telecommun. Syst..