Discrete Time Markov Chains with Actions and Nondeterminism

In this chapter we introduce some advanced probabilistic models that can be defined by enriching the transition functions of PTSs. As we have seen for Markov chains, the transition system representation is very useful since it comes with a notion of bisimilarity. In fact, using the advanced, categorical notion of coalgebra, which however we will not develop further, there is a standard method to define bisimilarity just according to the type of the transition function. Also a corresponding notion of Hennessy-Milner logic can be defined accordingly. First we will see two different ways to add observable actions to our probabilistic models, then we will present extensions which combine nondeterminism, actions and probabilities.