Classification Problems in MDPs

In this paper we investigate classification problems for Markov deci-sion processes (MDPs). These MDPs can be classified in several ways. One way is based on the concept communicating, and distinguishes between communicating, weakly communicating and noncommunicating. Another way of classification is based on the ergodic structure. In this approach the distinction between completely ergodic, unichain and multichain is made. Furthermore, there is a classification based on decomposition of the state space. This decomposition distinguishes between several levels. At each level there is a set of recurrent classes and a (perhaps empty) set of transient states.