Discrete Stochastic Processes [Book Reviews]

Discrete stochastic processes, as loosely defined in the book, are stochastic processes such that " interesting things (arrivals, departures, changes of state) occur at discrete instants of time separated by deterministic or random intervals. " Included in this definition are thus not only discrete-time random processes, such as Markov chains, random walks, branching processes, and martingale sequences, but also jump-type continuous-time random processes such as the Poisson process, renewal processes, and countable-state Markov processes. The focus of the book is unabashedly on helping the student gain a good intuitive understanding of the basic theory. This is reflected in prose and problems that require the student to think, rather than just follow recipes. It is also reflected in the absence of the technical background which is necessary for a firm mathematical underpinning (for example, sigma algebras are not explicitly covered in the book) but which are not essential for a solid intuitive understanding of discrete stochastic processes. The book is not devoid of rigorous proofs, in spite of the omission of the basic mathematical underpinnings. For example, basic inequalities for martingales are proved and used to prove the strong law of large numbers. Also, Blackwell's Renewal Theorem is stated (without proof but with ample intuitive justification and proof for a special case) and used to derive the basic classification results for Markov chains and countable state Markov processes. There is a smattering of applied materialfor example, both elementary queueing theory and hypothesis testing are covered. Incidentally, this juxtaposition allows the author to point out that Wald's inequality associated with sequential hypothesis testing is, in the context of queueing theory, Kingman's exponential bound on the waiting time in a G/G/1 queue. There is also material related to optimization, such as Markov decision theory and dynamic programming. The student should not be expected to pick up these application areas from this book alone, but the book can serve either as an introduction for students not familiar with the various applications, or as a tie back to basic theory for students that already have familiarity with the applications. One of the distinguishing features of the book is the excellent use of diagrams to help explain basic concepts and proofs. Another feature of the book is that it looks at some of the basic random processes from several different angles, and sometimes discusses the relative merits of the different points of view. For example, there …