Multivariate Autoregressive Mixture Models for Music Auto-Tagging

We propose the multivariate autoregressive model for content based music auto-tagging. At the song level our approach leverages the multivariate autoregressive mixture (ARM) model, a generative time-series model for audio, which assumes each feature vector in an audio fragment is a linear function of previous feature vectors. To tackle tagmodel estimation, we propose an efficient hierarchical EM algorithm for ARMs (HEM-ARM), which summarizes the acoustic information common to the ARMs modeling the individual songs associated with a tag. We compare the ARM model with the recently proposed dynamic texture mixture (DTM) model. We hence investigate the relative merits of different modeling choices for music time-series: i) the flexibility of selecting higher memory order in ARM, ii) the capability of DTM to learn specific frequency basis for each particular tag and iii) the effect of the hidden layer of the DT versus the time efficiency of learning and inference with fully observable AR components. Finally, we experiment with a support vector machine (SVM) approach that classifies songs based on a kernel calculated on the frequency responses of the corresponding song ARMs. We show that the proposed approach outperforms SVMs trained on a different kernel function, based on a competing generative model.