Learning the Features ignored by Classification Models using Staged Modeling Variational Autoencoder

Classification networks usually only save the specified factor associated with labels. We propose a probabilistic graphical model (PGM) to learn the features ignored by arbitrary classification networks using staged modeling. To implement the staged PGM, we introduce Staged Modeling Variational Autoencoder (SMVAE), in which the first stage can apply arbitrary classification models to encode the specified factor, then optimizing the Evidence Lower Bound (ELBO) given optimal specified factor to compress the features ignored at the first stage into the unspecified factor. Besides, SMVAE can learn the disentangled unspecified factors unsupervised by further decomposing the ELBO given the optimal specified factor. At last, we introduce Adain based on Infusion Training in SMVAE to learn more details to reconstruct data. Detailed experiments are given to evaluate the disentanglement and performance of SMVAE.