Random Forest Classifiers

A classification tree represents the probability spaceP of posterior probabilities p(y|x) of label given feature by a recursive partition of the feature space X , where each partition is performed by a test on the feature x. Each such test is called a split rule. Since the partition is recursive, the split rules can be arranged into a tree. The split rules are learned by partitioning the training set T recursively in a way that increases the purity of the subsets formed by each split. A set is pure if one of the labels dominate the others in the set, in a sense to be made more precise later on. To each set of the partition is assigned a posterior probability distribution, and p(y|x) for a training or test feature x ∈ X is then defined as the probability distribution associated with the set of the partition that contains x. A popular binary split rule called a 1-rule partitions a subset1 S ⊆ X × Y into the two sets