Mondrian Confidence Machine

Mondrian Confidence Machine (MCM) is an on-line prediction algorithm that, given a split of all examples into a finite number of types k and for each type a significance level δk, outputs as its prediction the set of labels deemed possible at the level δk. MCM includes as special cases Transductive Confidence Machine (TCM) and Inductive Confidence Machine (ICM) and is designed to take care of such issues as different risks of false positive and false negative predictions, conditional inference, and a slow teacher. In this paper we generalize known results about TCM and ICM showing that each MCM is type-wise well-calibrated, in the sense that predictions at significance levels δk will be wrong with relative frequency at most δk for each type k in the long run. Our experimental results show advantages of MCM over the previously known algorithms.