Applied Probability Models

1. REVIEW OF PROBABILITY THEORY Experiments and Outcomes / Sample Space and Events / Probabilities Defined on Events / Probability Models / Probabilities of Combined Events / Conditional Probabilities / Conditional Arguments / Independent Events / Random Variables / Discrete Random Variables and Their Expected Values / Bernoulli Random Variables / Geometric Distribution / Binomial Distribution / Negative Binomial Distribution / Continuous and Mixed Random Variables / Continuous Random Variables and Their Expected Values / Uniform Distribution / Normal Distribution / Conditional Arguments for a Random Variable / Conditional Expectations / Random Sum / Summary / Problems 2. DISCRETE-TIME MARKOV CHAINS Stochastic Processes / Independent and Identically Distributed Random Variables / Markov Chains of Order i / First-Order Markov Chains / Matrix Notation for Markov Chains / Transition Diagrams / The Importance of First-Order Markov Chains / Time-Homogeneity / Seasonal Chains / The Rat in the Maze / Empirical Transition Matrices / Population-Homogeneity / Enlarging the State Space / Using Supplementary Variables / Refining the State Space / What Do a Little Rat and a Big Company Have in Common? / Lumpable States / Discrete-Time Chains In a Continuous-Time Chain / The Sojourn Times / Summary / Problems 3. TRANSIENT AND LIMITING RESULTS Paths / k-Step Transition Probabilities / Path Analysis / Matrix Multiplication Method / Chapman-Kolmogorov Equation / Initial Conditions / Transient Results / Validating the Model / Limiting Distributions / Expected Numbers of Visits / Summary / Problems 4. CLASSIFICATION OF FINITE CHAINS Classification of Finite Chains / Transient and Recurrent States / Reachability Between States / Communication of Two States / Classes / Transient and Recurrent Classes / Canonical Forms / Absorbing Chains / Irreducible Chains / Periodic States / Periodic Classes / Finite, Periodic Chains / Summary / Problems 5. FINITE ABSORBING CHAINS Absorbing Chains / First-Step Analysis / Expected Number of Visits to a Transient State / Expected Absorption Time / Absorbing Probabilities / Limiting Distributions / Summary / Problems 6. FINITE NON-ABSORBING CHAINS Non-Absorbing Chains / The Taxicab Example / Limiting Probabilities / Long-term Visiting Rates / Expected First Reaching Times / Expected Return Times / Regenerative Property / Periodic Chains / Finite Reducible Chains / Transient Behavior / Class-Absorption Probabilities / Long-Term Visiting Rate / Limiting Distribution / Summary / Problems 7. INFINITE CHAINS Infinite Random Walks / Infinite Chains / Infinite Absorbing Chains / Infinite Irreducible Chains / Positive-Recurrence / Null-Recurrence / Summary / Problems 8. POISSON STREAMS OF EVENTS Continuous-Time Chains / Markovian Streams / Regular Streams / Poisson Streams / Instantaneous and Average Rates / Rates / Remaining Lives / Exponential and Hyperexponential Distributions / Lives / The Shorter Remaining Life / Poisson (Counting) Process / Erlang Distributions / Random Streams / Superposition of Two Poisson Streams / Decomposition of a Poisson Stream / Compound Poisson Process / Summary / Problems 9. RENEWAL STREAMS OF EVENTS Renewal Streams and Renewal (Counting) Processes / Elementary Renewal Theorem / Renewal Rates / Renewal-Reward Processes / Gradual-Reward Processes / Regenerative Processes / Remaining Lives / Lattice Random Variables / Spread-Out Random Variables / Limiting Distributions / Summary / Problems 10. SEMI-MARKOV CHAINS Semi-Markov Chains / The Rat Maze Example / A Marketing Example / Holding Times / Sojourn Times / Classification of States / Absorbing Semi-Markov Chains / Irreducible Semi-Markov Chains / Expected First Reaching Times / Expected Return Times / Visiting Rates / Limiting Results / Summary / Problems 11. CONTINUOUS-TIME MARKOV CHAINS Continuous-Time Markov Chain / Memoryless Property / Transition Rates / Transition Diagrams / The Q-Matrix / Equally-Spaced Discrete-Time Chains / Why This Model? / Competitive Chains / The Machine-Repair Example / The Balance Equations / Birth-and-Death Processes / Summary / Problems 12. MARKOVIAN QUEUES Queuing Theory / Queuing Systems / Kendall Notation / Performance Measures / Little's Result / Markovian Queues / M/M/k/N Queues / M/M/k Queues / M/M/k Queues with Feedback / M[G]/M/1 Queues / M/M/2 Queues with Different Service Rates / M/M/1 Queues with N-Policy / M[G]/M/1 Queues / Queues in Series with Blocking / Queues in Series without Blocking / Acyclic Open Jackson Networks / General Open Jackson Networks / Summary / Problems 13. GENERAL SINGLE-SERVER QUEUES G/G/1 Queues / Busy Cycles / Waiting Times and Idle Times / Model Approximation: M/G/1 Queues / Analytical Approximations / Numerical Methods / Simulation / Random Number Generators / Random Variate Generators / Inverse Transform Technique for the Discrete Distributions / Inverse Transform Technique for the Continuous Distributions / Acceptance-Rejection Technique for the Discrete Distributions / Acceptance-Rejection Technique for the Continuous Distributions / Generating Specific Distributions / Simulating E^2/ E^2/1 Queues / Simulation and Analytical Methods / Summary / Problems