Exploring Music Video Editing Rules with Dual-Wing Harmonium Model

Automatic music video editing is still a challenging task due to the lack of knowledge of how music and video are matched to produce attractive effects. Previous works usually matches music and video following assumption or empirical knowledge. In this paper, we use a dual-wing harmonium model to learn and represent the underlying music video editing rules from a large dataset of music videos. The editing rules are extracted by clustering the low dimensional representation of music video clips. In the experiments, we give an intuitive visualization for the discovered editing rules. These editing rules partially reflect professional music video editor's skills and can be used to further improve the quality of automatically generated music video.