Horizontal and Vertical Ensemble with Deep Representation for Classification

Representation learning, especially which by using deep learning, has been widely applied in classication. However, how to use limited size of labeled data to achieve good classication performance with deep neural network, and how can the learned features further improve classication remain indenite. In this paper, we propose Horizontal Voting Vertical Voting and Horizontal Stacked Ensemble methods to improve the classication performance of deep neural networks. In the ICML 2013 Black Box Challenge, via using these methods independently, Bing Xu achieved 3rd in public leaderboard, and 7th in private leaderboard; Jingjing Xie achieved 4th in public leaderboard, and 5th in private leaderboard.