Examples and Survey of Applied Perturbed Stochastic Models

In Chapter 6, we present a background for construction of numerical examples for asymptotic results presented in Chapters 2– 5, some numerical examples as well as a summary survey of applied perturbed stochastic systems.

[1]  K. Avrachenkov,et al.  Singular perturbations of Markov chains and decision processes , 2002 .

[2]  Ivo Marek,et al.  Convergence issues in the theory and practice of iterative aggregation/disaggregation methods. , 2009 .

[3]  M. Petersson.  Asymptotics for Quasi-stationary Distributions of Perturbed Discrete Time Semi-Markov Processes , 2016, 1603.05895.

[4]  A. Swishchuk,et al.  Semi-Markov Random Evolutions , 1994 .

[5]  David S. Kim,et al.  An exact aggregation/disaggregation algorithm for large scale markov chains , 1995 .

[6]  P. J. Courtois.  Decomposability of Queueing Networks , 1977 .

[7]  Mats Gyllenberg,et al.  Quasi-Stationary Phenomena in Nonlinearly Perturbed Stochastic Systems , 2007 .

[8]  Perturbed Markov Processes , 2003 .

[9]  Ying Ni.  Exponential Asymptotical Expansions for Ruin Probability in a Classical Risk Process with Non-polynomial Perturbations , 2014 .

[10]  Mats Gyllenberg,et al.  Quasi-Stationary Phenomena for Semi-Markov Processes , 1999 .

[11]  Konstantin Avrachenkov,et al.  Analytic Perturbation Theory and Its Applications , 2013 .

[12]  Ward Whitt,et al.  An Introduction to Stochastic-Process Limits and their Application to Queues , 2002 .

[13]  Sergei Silvestrov,et al.  The Mathematics of Internet Search Engines , 2008 .

[14]  William S. Griffith,et al.  athematical Theory of Reliability of Time Dependent Systems With Practical Applications , 1999, Technometrics.

[15]  A. A. Pervozvanskiĭ,et al.  Stationary-state evaluation for a complex system with slowly varying couplings , 1974 .

[16]  Iterative aggregation: Convergence rate , 1984 .

[17]  A. Borovkov.  Ergodicity and stability of stochastic processes , 1998 .

[18]  E. Seneta,et al.  Finite approximations to infinite non-negative matrices, II: refinements and applications , 1968, Mathematical Proceedings of the Cambridge Philosophical Society.

[19]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .

[20]  F. Delebecque.  A Reduction Process for Perturbed Markov Chains , 1983 .

[21]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[22]  O. Hossjer,et al.  Nonlinearly Perturbed Birth-Death-Type Models , 2016, 1604.02295.

[23]  Jeffrey J. Hunter,et al.  Mixing times with applications to perturbed Markov chains , 2006 .

[24]  J. Hunter.  Generalized inverses of Markovian kernels in terms of properties of the Markov chain , 2012, 1209.3533.

[25]  V. Kalashnikov,et al.  Geometric Sums: Bounds for Rare Events with Applications: Risk Analysis, Reliability, Queueing , 1997 .

[26]  Approximation of eigencharacteristics in nearly-completely decomposable stochastic systems , 1976 .

[27]  K. Avrachenkov,et al.  The first Laurent series coefficients for singularly perturbed stochastic matrices , 2004 .

[28]  Christopher Engström.  PageRank in Evolving Networks and Applications of Graphs in Natural Language Processing and Biology , 2016 .

[29]  L. Hörmander,et al.  An introduction to complex analysis in several variables , 1973 .

[30]  Kyle W. Kindle,et al.  Iterative aggregation for solving undiscounted semi-markovian reward processes , 1986 .

[31]  Walter L. Smith,et al.  Regenerative stochastic processes , 1955, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[32]  Carl D. Meyer,et al.  Stochastic Complementation, Uncoupling Markov Chains, and the Theory of Nearly Reducible Systems , 1989, SIAM Rev..

[33]  S. Sastry,et al.  Hierarchical aggregation of singularly perturbed finite state Markov processes , 1983 .

[34]  M. Petersson.  Quasi-Stationary Distributions for Perturbed Discrete Time Regenerative Processes , 2015 .

[35]  Eugene Seneta,et al.  Finite approximations to infinite non-negative matrices , 1967, Mathematical Proceedings of the Cambridge Philosophical Society.

[36]  Beatrice Meini,et al.  Numerical Methods for Structured Markov Chains (Numerical Mathematics and Scientific Computation) , 2005 .

[37]  G. Stewart.  Gaussian Elimination, Perturbation Theory, and Markov Chains , 1993 .

[38]  Françoise Chatelin,et al.  Aggregation/Disaggregation for Eigenvalue Problems , 1984 .

[39]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[40]  Moshe Haviv.  On censored Markov chains, best augmentations and aggregation/disaggregation procedures , 1999, Comput. Oper. Res..

[41]  Yuriy E. Obzherin,et al.  Semi-Markov Models: Control of Restorable Systems with Latent Failures , 2015 .

[42]  N. Limnios,et al.  Semi-Markov Processes and Reliability , 2012 .

[43]  Eugene Seneta.  The principle of truncations in applied probability , 1968 .

[44]  Paul J. Schweitzer,et al.  Iterative Aggregation-Disaggregation Procedures for Discounted Semi-Markov Reward Processes , 1985, Oper. Res..

[45]  P. Kokotovic,et al.  A singular perturbation approach to modeling and control of Markov chains , 1981 .

[46]  Ingemar Nåsell.  Extinction and Quasi-stationarity in the Stochastic Logistic SIS Model Introduction , 2011 .

[47]  Nonlinearly Perturbed Stochastic Processes and Systems , 2010 .

[48]  G. Stewart,et al.  The Laurent expansion of pencils that are singular at the origin , 1991 .

[49]  Nikolaos Limnios,et al.  Stochastic Systems in Merging Phase Space , 2005 .

[50]  R. Lande,et al.  Stochastic Population Dynamics in Ecology and Conservation , 2003 .

[51]  Ying Ni.  Analytical and Numerical Studies of Perturbed Renewal Equations with Multivariate Non-Polynomial Perturbations , 2010 .

[52]  E. A. Doorn.  Stochastic Monotonicity and Queueing Applications of Birth-Death Processes , 1981 .

[53]  Boris Harlamov,et al.  Continuous Semi-Markov Processes , 2008 .

[54]  Konstantin Avrachenkov,et al.  On transition matrices of Markov chains corresponding to Hamiltonian cycles , 2014, Annals of Operations Research.

[55]  Samuel Karlin,et al.  ELEMENTS OF STOCHASTIC PROCESSES , 1975 .

[56]  Sergei Silvestrov,et al.  Generalisation of the Damping Factor in PageRank for Weighted Networks , 2014 .

[57]  G. Yin,et al.  Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications , 2004 .

[58]  Pierre Collet,et al.  Quasi-stationary distributions , 2011 .

[59]  Sergei Silvestrov,et al.  PageRank, a Look at Small Changes in a Line of Nodes and the Complete Graph , 2016 .

[60]  Ying Ni.  Nonlinearly Perturbed Renewal Equations : asymptotic Results and Applications , 2011 .

[61]  P. Schweitzer.  Dual bounds on the equilibrium distribution of a finite Markov chain , 1987 .

[62]  Maria Rieders,et al.  A New Algorithm for Computing the Ergodic Probability Vector for Large Markov Chains , 1990, Probability in the Engineering and Informational Sciences.

[63]  Moshe Haviv,et al.  On Series Expansions and Stochastic Matrices , 1993, SIAM J. Matrix Anal. Appl..

[64]  Hendrik Vantilborgh,et al.  Aggregation with an error of O(ε2) , 1985, JACM.

[66]  B. L. Miller,et al.  Discrete Dynamic Programming with a Small Interest Rate , 1969 .

[67]  Dmitrii Silvestrov.  Limit theorems for randomly stopped stochastic processes , 2004 .

[68]  An approximation to the stationary distribution of a nearly completely decomposable Markov chain and its error bound , 1986 .

[69]  Qing Zhang,et al.  Discrete-time dynamic systems arising from singularly perturbed Markov chains , 2001 .

[70]  R. Manca,et al.  Reward Algorithms for Semi-Markov Processes , 2016, 1603.05693.

[71]  D. Silvestrov,et al.  Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes , 2016 .

[72]  Mohammed Abbad,et al.  Algorithms for Singularly Perturbed Markov Control Problems: A Survey11We are indebted to M. Haviv for a number of helpful discussionsand for his comments on anearlier draft of the manuscript. , 1995 .

[73]  Moshe Haviv,et al.  An aggregation/disaggregation algorithm for computing the stationary distribution of a large markov chain , 1992 .

[74]  Jean B. Lasserre,et al.  The fundamental matrix of singularly perturbed Markov chains , 1999, Advances in Applied Probability.

[75]  William J. Stewart,et al.  Iterative aggregation/disaggregation techniques for nearly uncoupled markov chains , 1985, JACM.

[76]  Uriel G. Rothblum,et al.  Taylor expansions of eigenvalues of perturbed matrices with applications to spectral radii of nonnegative matrices , 1992 .

[77]  D. Blackwell.  Discrete Dynamic Programming , 1962 .

[78]  Mats Gyllenberg,et al.  Nonlinearly perturbed regenerative processes and pseudo-stationary phenomena for stochastic systems , 2000 .

[79]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[80]  G. Stewart,et al.  On a direct method for the solution of nearly uncoupled Markov chains , 1991 .

[81]  On the perturbation of Markov chains with nearly transient states , 1992 .

[82]  Nikolaos Limnios,et al.  Semi-Markov Chains and Hidden Semi-Markov Models toward Applications: Their Use in Reliability and DNA Analysis , 2008 .

[83]  J. Janssen,et al.  Semi-Markov Risk Models for Finance, Insurance and Reliability , 2007 .

[84]  M. Petersson.  Asymptotics of Ruin Probabilities for Perturbed Discrete Time Risk Processes , 2014 .

[85]  A. F. Veinott.  Discrete Dynamic Programming with Sensitive Discount Optimality Criteria , 1969 .

[86]  K. Avrachenkov,et al.  Perturbation of null spaces with application to the eigenvalue problem and generalized inverses , 2003 .

[87]  Asymptotic expansions and inequalities in stability theorems for general Markov chains with relatively bounded perturbations , 1988 .

[88]  I. Marek,et al.  A note on local and global convergence analysis of iterative aggregation–disaggregation methods , 2006 .

[89]  Refael Hassin,et al.  Mean Passage Times and Nearly Uncoupled Markov Chains , 1992, SIAM J. Discret. Math..

[90]  S. Silvestrov,et al.  PageRank, Connecting a Line of Nodes with a Complete Graph , 2016 .

[91]  M. Haviv.  Aggregation/disaggregation methods for computing the stationary distribution of a Markov chain , 1987 .

[92]  Vladimir V. Anisimov,et al.  Switching Processes in Queueing Models , 2008 .

[93]  Asymptotic Expansions for Moment Functionals of Perturbed Discrete Time Semi-Markov Processes , 2016, 1603.05891.

[94]  P.-J. Courtois,et al.  Decomposability: Queueing and Computer System Applications , 2014 .

[95]  Daryl J. Daley,et al.  Epidemic Modelling: An Introduction , 1999 .

[96]  Refael Hassin,et al.  To Queue or Not to Queue: Equilibrium Behavior in Queueing Systems , 2002 .

[97]  Qing Zhang,et al.  Discrete-time Singularly Perturbed Markov Chains , 2003 .

[98]  M. Petersson.  Perturbed discrete time stochastic models , 2016 .

[99]  Vladimir V. Kalashnikov,et al.  Mathematical Methods in Queuing Theory , 1993 .

[100]  Samuel S. Chiu,et al.  A Method to Calculate Steady-State Distributions of Large Markov Chains by Aggregating States , 1987, Oper. Res..

[101]  Ying Ni.  NONLINEARLY PERTURBED RENEWAL EQUATIONS : THE NON-POLYNOMIAL CASE , 2012 .

[102]  V. Borkar,et al.  Hamiltonian Cycle Problem and Markov Chains , 2012 .

[103]  Corina E. Tarnita,et al.  Measures of success in a class of evolutionary models with fixed population size and structure , 2014, Journal of mathematical biology.

[104]  P. Collet,et al.  Quasi-Stationary Distributions: Markov Chains, Diffusions and Dynamical Systems , 2012 .

[105]  P. Schweitzer.  Perturbation theory and finite Markov chains , 1968 .

[106]  G. Louchard,et al.  Return times in nearly completely decomposable stochastic processes , 1978 .

[107]  Mats Gyllenberg,et al.  Single-species metapopulation dynamics: A structured model , 1992 .

[108]  A. Hanen.  Théorèmes limites pour une suite de chaînes de Markov , 1963 .

[109]  U. Sumita,et al.  A New Algorithm For Computing The Ergodic Probability Vector For Large Markov Chains: Replacement Process Approach , 1988 .

[110]  K. Avrachenkov.  Singularly Perturbed Finite Markov Chains with General Ergodic Structure , 2000 .