Increasing Convex Monotone Markov Chains: Theory, Algorithm and Applications

We develop theoretical and algorithmic aspects of discrete-time Markov chain comparison with the increasing convex order. This order is based on the variability of the process and it is expected that one can get more accurate bounds with such an order although the monotonicity property is more complex. We give a characterization for finite state space to obtain an algebraic description which is suitable for an algorithmic framework. We develop an algorithm and we introduce some applications related to the worst case stochastic analysis when some high level information is known, but not the complete structure of the chain.

[1]  Richard E. Barlow,et al.  Statistical Theory of Reliability and Life Testing: Probability Models , 1976 .

[2]  J. Keilson,et al.  Monotone matrices and monotone Markov processes , 1977 .

[3]  Pierre Semal,et al.  Bounds for the Positive Eigenvectors of Nonnegative Matrices and for their Approximations by Decomposition , 1984, JACM.

[4]  Pierre Semal,et al.  Computable Bounds for Conditional Steady-State Probabilities in Large Markov Chains and Queueing Models , 1986, IEEE J. Sel. Areas Commun..

[5]  Michel Doisy,et al.  Comparaison de processus markoviens , 1992 .

[6]  Marco Scarsini,et al.  Stochastic Orders and Applications: A Classified Bibliography , 1993 .

[7]  Geert Jan Olsder,et al.  Synchronization and Linearity: An Algebra for Discrete Event Systems , 1994 .

[8]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[9]  Moshe Shaked,et al.  Stochastic orders and their applications , 1994 .

[10]  Moshe Shaked,et al.  Stochastic Convexity and Concavity of Markov Processes , 1994, Math. Oper. Res..

[11]  Nihal Pekergin,et al.  Stochastic Performance Bounds by State Space Reduction , 1999, Perform. Evaluation.

[12]  Laurent Truffet.  Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons , 2000, Journal of Applied Probability.

[13]  James Ledoux,et al.  Markovian bounds on functions of finite Markov chains , 2001 .

[14]  N. Pekergin,et al.  CLOSED-FORM STOCHASTIC BOUNDS ON THE STATIONARY DISTRIBUTION OF MARKOV CHAINS , 2002, Probability in the Engineering and Informational Sciences.

[15]  Jean-Michel Fourneau,et al.  An Algorithmic Approach to Stochastic Bounds , 2002, Performance.

[16]  Taizhong Hu,et al.  Moment Inequalities for Discrete Ageing Families , 2003 .

[17]  Miklós Telek,et al.  Acyclic discrete phase type distributions: properties and a parameter estimation algorithm , 2003, Perform. Evaluation.

[18]  Mark A. McComb.  Comparison Methods for Stochastic Models and Risks , 2003, Technometrics.

[19]  Stephen Gilmore,et al.  PEPA nets: a structured performance modelling formalism , 2002, Perform. Evaluation.

[20]  A. Bobbioa,et al.  Acyclic discrete phase type distributions: properties and a parameter estimation algorithm , 2003 .

[21]  Jean-Michel Fourneau,et al.  Transforming stochastic matrices for stochastic comparison with the st-order , 2003, RAIRO Oper. Res..

[22]  Jean-Michel Fourneau,et al.  An open tool to compute stochastic bounds on steady-state distributions and rewards , 2003, 11th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer Telecommunications Systems, 2003. MASCOTS 2003..

[23]  J. Fourneau,et al.  Algorithms for an irreducible and lumpable strong stochastic bound , 2004 .

[24]  Ana Busic,et al.  A Matrix Pattern Compliant Strong Stochastic Bound , 2005, 2005 Symposium on Applications and the Internet Workshops (SAINT 2005 Workshops).

[25]  Peter Buchholz,et al.  An improved method for bounding stationary measures of finite Markov processes , 2005, Perform. Evaluation.

[26]  Nihal Pekergin,et al.  Model Checking of Continuous-Time Markov Chains by Closed-Form Bounding Distributions , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[27]  Ana Busic,et al.  Worst Case Analysis of Batch Arrivals with the Increasing Convex Ordering , 2006, EPEW.

[28]  Serge Haddad,et al.  Sub-stochastic matrix analysis for bounds computation - Theoretical results , 2007, Eur. J. Oper. Res..