Accelerate convolutional neural networks for binary classification via cascading cost-sensitive feature

Convolutional Neural Networks (CNNs) have delivered impressive state-of-the-art performances for many vision tasks, while the computation costs of these networks during test-time are notorious. Empirical results have discovered that CNNs have learned the redundant representations both within and across different layers. When CNNs are applied for binary classification, we investigate a method to exploit this redundancy across layers, and construct a cascade of classifiers which explicitly balances classification accuracy and hierarchical feature extraction costs. Our method cost-sensitively selects feature points across several layers from trained networks and embeds non-expensive yet discriminative features into a cascade. Experiments on binary classification demonstrate that our framework leads to drastic test-time improvements, e.g., possible 47.2x speedup for TRECVID upper body detection, 2.82x speedup for Pascal VOC2007 People detection, 3.72x for INRIA Person detection with less than 0.5% drop in accuracies of the original networks.