Rademacher Complexity Bounds for a Penalized Multi-class Semi-supervised Algorithm (Extended Abstract)

We propose Rademacher complexity bounds for multiclass cla sifiers trained with a two-step semi-supervised model. In the first s tep, the algorithm partitions the partially labeled data and then identifies dense clusters containingκ predominant classes using the labeled training examples such t hat the proportion of their non-predominant classes is below a fixed threshold. In the se cond step, a classifier is trained by minimizing a margin empirical loss over the lab eled training set and a penalization term measuring the disability of the learner t o predict theκ predominant classes of the identified clusters. The resulting data-depe ndent generalization error bound involves the margin distribution of the classifier, th e stability of the clustering technique used in the first step and Rademacher complexity te rms corresponding to partially labeled training data. Our theoretical result ex hibit convergence rates extending those proposed in the literature for the binary case , nd experimental results on different multiclass classification problems show empir ical evidence that supports the theory.