Effect of parameter sharing for multimodal deep autoencoders

Partial observation can be avoided by extracting both modality specific features and common features from multimodal data. This paper proposes a framework of parameter shared multimodal deep autoencoders which uses complemental multimodal data in order to learn both modality specific and common features. The proposed model shares parameters of networks for each modality, while conventional multimodal deep autoencoder models share top layer neurons of their encoder among the modalities. The parameters of the proposed networks are shared by minimizing the Frobenius norm between those parameters. In experiments, we test the proposal on a classification task with complemental multimodal data. Experiment results show that our framework enables to learn specific and common features of the multimodal data.