A feasibility study of an autoencoder meta-model for improving generalization capabilities on training sets of small sizes

A meta-model approach for autoencoders is presented.New modification of autoencoder's learning algorithm is proposed.Developed meta-model provides faster training in terms of consumed training samples. The problem of training autoencoders (with logistic regression as the classification layer) on sets of small sizes is considered on the example of image classification and scene categorization tasks. Conventional autoencoders with uniform priors usually fail to learn useful features from few samples. A possibility to overcome this difficulty is considered on the example of a proposed meta-model generating autoencoders from a vector of meta-parameters of much smaller dimension than the number of autoencoder parameters. Handcrafted image features designed for the task of scene classification are used as the baseline for comparison. Unbiased autoencoders showed the worst results on tiny training sets and seem to require much larger sets to outperform other methods. On the other hand, the developed biased autoencoders work better on the training sets of small sizes, but have much less discriminative power, since the considered meta-model assigns strictly zero probabilities to a large subset of solutions. General possibility of increasing training speed (in term of consumed training patterns) of autoencoders is confirmed.