Construction and solution of Markov reward models

Stochastic Petri nets (SPNs) and extensions are a popular method for evaluating a wide variety of systems. In most cases, the interesting measures regarding the system's characteristics can be de ned at the net level by means of reward variables. Depending on the measures, these net-level reward models are solved either by rst generating a statelevel reward model or by directly generating paths from the net-level description. In this thesis, we propose algorithms for the generation of state-level reward models as well as for directly obtaining solutions from net-level reward models when the net-level reward models are speci ed as stochastic activity networks (SANs) with \step-based reward structure". Moreover, we propose algorithms for computing the expected value and the probability distribution function of a reward variable at speci ed time instants, and for computing the probability distribution function of reward accumulated during a nite interval. The interval may correspond to the mission period in a mission-critical system, the time between scheduled maintenances, or a warranty period; whereas the time instants may be critical instances during these intervals. The proposed algorithms avoid the construction of statelevel representations and the memory growth problems experienced when applying previous approaches to large models. Furthermore, we study the e ect of workload on the availability and response time of voting algorithms. Voting algorithms are a popular way to provide data consistency

[1]  Philip Heidelberger,et al.  Fast simulation of dependability models with general failure, repair and maintenance processes , 1990, [1990] Digest of Papers. Fault-Tolerant Computing: 20th International Symposium.

[2]  Lorenzo Donatiello,et al.  Analysis of a composite performance reliability measure for fault-tolerant systems , 1987, JACM.

[3]  Mostafa H. Ammar,et al.  The grid protocol: a high performance scheme for maintaining replicated data , 1990, [1990] Proceedings. Sixth International Conference on Data Engineering.

[4]  William H. Sanders,et al.  Reduced Base Model Construction Methods for Stochastic Activity Networks , 1991, IEEE J. Sel. Areas Commun..

[5]  Akhil Kumar,et al.  Performance analysis of a hierarchical quorum consensus algorithm for replicated objects , 1990, Proceedings.,10th International Conference on Distributed Computing Systems.

[6]  Donald Gross,et al.  The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes , 1984, Oper. Res..

[7]  David H. Bailey,et al.  Multiprecision Translation and Execution of Fortran Programs , 1993 .

[8]  Louise E. Moser,et al.  Probabilistic language analysis of weighted voting algorithms , 1990, SIGMETRICS '90.

[9]  Divyakant Agrawal,et al.  The generalized tree quorum protocol: an efficient approach for managing replicated data , 1992, TODS.

[10]  Gianfranco Ciardo,et al.  A Characterization of the Stochastic Process Underlying a Stochastic Petri Net , 1994, IEEE Trans. Software Eng..

[11]  Kishor S. Trivedi,et al.  Numerical transient analysis of markov models , 1988, Comput. Oper. Res..

[12]  Daniel P. Siewiorek,et al.  Workload, Performance, and Reliability of Digital Computing Systems. , 1980 .

[13]  Lu Wei,et al.  Analysis of workload influence on dependability , 1988, [1988] The Eighteenth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[14]  Philip Heidelberger,et al.  Uniformization and exponential transformation: Techniques for fast simulation of highly dependable non-Markovian systems , 1992, [1992] Digest of Papers. FTCS-22: The Twenty-Second International Symposium on Fault-Tolerant Computing.

[15]  Sushil Jajodia,et al.  Dynamic voting algorithms for maintaining the consistency of a replicated database , 1990, TODS.

[16]  Gianfranco Ciardo,et al.  Stochastic Petri Net Analysis of a Replicated File System , 1989, IEEE Trans. Software Eng..

[17]  A. Jensen,et al.  Markoff chains as an aid in the study of Markoff processes , 1953 .

[18]  William H. Sanders,et al.  Performability modeling with UltraSAN , 1991, IEEE Software.

[19]  William H. Sanders,et al.  Importance Sampling Simulation in UltraSAN , 1994, Simul..

[20]  William H. Sanders,et al.  METASAN: A Performability Evaluation Tool Based on Stochastic Acitivity Networks , 1986, FJCC.

[21]  Christoph Lindemann An Improved Numerical Algorithm for Calculating Steady-State Solutions of Deterministic and Stochastic Petri Net Models , 1993, Perform. Evaluation.

[22]  William H. Sanders,et al.  Dependability Evaluation Using Composed SAN-Based Reward Models , 1992, J. Parallel Distributed Comput..

[23]  Asser N. Tantawi,et al.  Evaluation of Performability for Degradable Computer Systems , 1987, IEEE Transactions on Computers.

[24]  Christoph Lindemann,et al.  DSPNexpress: A Software Package for the Efficient Solution of Deterministic and Stochastic Petri Nets , 1993, Perform. Evaluation.

[25]  T. Matsunawa The exact and approximate distributions of linear combinations of selected order statistics from a uniform distribution , 1985 .

[26]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[27]  Edmundo de Souza e Silva,et al.  Calculating availability and performability measures of repairable computer systems using randomization , 1989, JACM.

[28]  Mostafa H. Ammar,et al.  Performance Characterization of Quorum-Consensus Algorithms for Replicated Data , 1989, IEEE Trans. Software Eng..

[29]  N. A. J. Hastings,et al.  Dynamic Probabilistic Systems: Vol. 1: Markov Models; Vol. II: Semi-Markov and Decision Processes , 1973 .

[30]  Kishor S. Trivedi,et al.  Markov and Markov reward model transient analysis: An overview of numerical approaches , 1989 .

[31]  Edmundo de Souza e Silva,et al.  State space exploration in Markov models , 1992, SIGMETRICS '92/PERFORMANCE '92.

[32]  Che-Liang Yang,et al.  Efficient computation of most probably states for communication networks with multimode components , 1989, IEEE Trans. Commun..

[33]  Hector Garcia-Molina,et al.  Increasing availability under mutual exclusion constraints with dynamic vote reassignment , 1989, TOCS.

[34]  William H. Sanders,et al.  Efficient simulation of hierarchical stochastic activity network models , 1993, Discret. Event Dyn. Syst..

[35]  G. Constantine Combinatorial theory and statistical design , 1987 .

[36]  Vincenzo Grassi,et al.  Performability Evaluation of Fault-Tolerant Satellite Systems , 1987, IEEE Trans. Commun..

[37]  Jérôme Collet,et al.  An algorithm in order to implement reliability exponential approximations , 1994 .

[38]  Markus Siegle Reduced Markov Models Of Parallel Programs With Replicated Processes , 1994, Proceedings. Second Euromicro Workshop on Parallel and Distributed Processing.

[39]  Kishor S. Trivedi,et al.  SPNP: stochastic Petri net package , 1989, Proceedings of the Third International Workshop on Petri Nets and Performance Models, PNPM89.

[40]  H. Weisberg,et al.  The Distribution of Linear Combinations of Order Statistics from the Uniform Distribution , 1971 .

[41]  Maurice Herlihy,et al.  Dynamic quorum adjustment for partitioned data , 1987, TODS.

[42]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[43]  B. P. Shah,et al.  Analytic solution of stochastic activity networks with exponential and deterministic activities , 1998 .

[44]  Marco Ajmone Marsan,et al.  Generalized Stochastic Petri Nets: A Definition at the Net Level and Its Implications , 1993, IEEE Trans. Software Eng..

[45]  Giovanni Chiola,et al.  A Software Package for the Analysis of Generalized Stochastic Petri Net Models , 1985, PNPM.

[46]  Marco Ajmone Marsan,et al.  On Petri nets with deterministic and exponentially distributed firing times , 1986, European Workshop on Applications and Theory of Petri Nets.

[47]  John Riordan,et al.  Introduction to Combinatorial Analysis , 1959 .

[48]  Bruce Daniel McLeod,et al.  Performance evaluation of N-processor Time Warp using stochastic activity networks , 1993 .

[49]  Janet Elaine Tvedt,et al.  Matrix representations and analytical solution methods for stochastic activity networks , 1990 .

[50]  Elmer E Lewis,et al.  Monte Carlo simulation of Markov unreliability models , 1984 .

[51]  Hermann Kopetz,et al.  Dependability: Basic Concepts and Terminology , 1992 .

[52]  Jehan-François Pâris,et al.  Voting with Witnesses: A Constistency Scheme for Replicated Files , 1986, ICDCS.

[53]  Peter W. Glynn,et al.  Simulation and analysis of highly reliable systems , 1990 .

[54]  David M. Nicol,et al.  Reliability analysis of complex models using SURE bounds , 1995 .

[55]  Edmundo de Souza e Silva,et al.  Calculating transient distributions of cumulative reward , 1995, SIGMETRICS '95/PERFORMANCE '95.

[56]  Edmundo de Souza e Silva,et al.  Calculating Cumulative Operational Time Distributions of Repairable Computer Systems , 1986, IEEE Transactions on Computers.

[57]  Ali Movaghar,et al.  Performability modeling with stochastic activity networks , 1985 .

[58]  Leonard Kleinrock,et al.  Queueing Systems: Volume I-Theory , 1975 .

[59]  Kishor S. Trivedi,et al.  On the Solution of GSPN Reward Models , 1991, Perform. Evaluation.

[60]  Günter Hommel,et al.  TimeNET: A Toolkit for Evaluating Non-Markovian Stochastic Petri Nets , 1995, Perform. Evaluation.

[61]  C. Loan,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix , 1978 .

[62]  Kishor S. Trivedi,et al.  Performability Analysis: Measures, an Algorithm, and a Case Study , 1988, IEEE Trans. Computers.

[63]  Kishor S. Trivedi,et al.  Automated Generation and Analysis of Markov Reward Models Using Stochastic Reward Nets , 1993 .

[64]  Robbert van Renesse,et al.  Voting with ghosts , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[65]  Stéphane Natkin,et al.  Searching best paths to worst states , 1991, Proceedings of the Fourth International Workshop on Petri Nets and Performance Models PNPM91.

[66]  W. Grassmann Transient solutions in Markovian queues : An algorithm for finding them and determining their waiting-time distributions , 1977 .

[67]  John F. Meyer,et al.  A Performability Solution Method for Degradable Nonrepairable Systems , 1984, IEEE Transactions on Computers.

[68]  Ignas G. Niemegeers,et al.  Performability Modelling Tools and Techniques , 1996, Perform. Evaluation.

[69]  Peter W. Glynn,et al.  Computing Poisson probabilities , 1988, CACM.

[70]  Krishna R. Pattipati,et al.  A Unified Framework for the Performability Evaluation of Fault-Tolerant Computer Systems , 1993, IEEE Trans. Computers.

[71]  Jian Tang,et al.  A scheme for maintaining consistency and availability of replicated files in a partitioned distributed system , 1989, [1989] Proceedings. Fifth International Conference on Data Engineering.

[72]  Kishor S. Trivedi,et al.  Reliability estimation of fault-tolerant systems: tools and techniques , 1990, Computer.

[73]  John F. Meyer,et al.  Closed-Form Solutions of Performability , 1982, IEEE Transactions on Computers.

[74]  William H. Sanders,et al.  Stochastic Activity Networks: Structure, Behavior, and Application , 1985, PNPM.

[75]  Kishor S. Trivedi,et al.  NUMERICAL EVALUATION OF PERFORMABILITY AND JOB COMPLETION TIME IN REPAIRABLE FAULT-TOLERANT SYSTEMS. , 1990 .

[76]  William H. Sanders,et al.  An environment for importance sampling based on stochastic activity networks , 1994, Proceedings of IEEE 13th Symposium on Reliable Distributed Systems.

[77]  William H. Sanders,et al.  Reward Model Solution Methods with Impulse and Rate Rewards: An Algorithm and Numerical Results , 1994, Perform. Evaluation.

[78]  Mostafa H. Ammar,et al.  Optimizing vote and quorum assignments for reading and writing replicated data , 1989, [1989] Proceedings. Fifth International Conference on Data Engineering.

[79]  E. D. S. E. Silva,et al.  An Algorithm to Calculate Transient Distributions of Cumulative Reward , 1998 .

[80]  Kishor S. Trivedi,et al.  Techniques and Tools for Reliability and Performance Evaluation: Problems and Perspectives , 1994, Computer Performance Evaluation.

[81]  John F. Meyer,et al.  On Evaluating the Performability of Degradable Computing Systems , 1980, IEEE Transactions on Computers.