Using weak supervision in learning Gaussian mixture models

The expectation maximization algorithm is a popular approach to learning Gaussian mixture models from unlabeled data. In addition to the unlabeled data, in many applications, additional sources of information such as a-priori knowledge of mixing proportions are also available. We present a weakly supervised approach, in the form of a penalized expectation maximization algorithm that uses a-priori knowledge to guide the model training process. The algorithm penalizes those models whose predicted mixing proportions have high divergence from the a-priori mixing proportions. We also present an extension to incorporate both labeled and unlabeled data in a semi-supervised setting. Systematic evaluations on several publicly available datasets show that the proposed algorithms outperforms the expectation maximization algorithm. The performance gains are particularly significant when the amount of unlabeled data is limited and in the presence of noise.