Improving the Separability of Deep Features with Discriminative Convolution Filters for RSI Classification

The extraction of activation vectors (or deep features) from the fully connected layers of a convolutional neural network (CNN) model is widely used for remote sensing image (RSI) representation. In this study, we propose to learn discriminative convolution filter (DCF) based on class-specific separability criteria for linear transformation of deep features. In particular, two types of pretrained CNN called CaffeNet and VGG-VD16 are introduced to illustrate the generality of the proposed DCF. The activation vectors extracted from the fully connected layers of a CNN are rearranged into the form of an image matrix, from which a spatial arrangement of local patches is extracted using sliding window strategy. DCF learning is then performed on each local patch individually to obtain the corresponding discriminative convolution kernel through generalized eigenvalue decomposition. The proposed DCF learning characterizes that a convolutional kernel with small size (e.g., 3 × 3 pixels) can be effectively learned on a small-size local patch (e.g., 8 × 8 pixels), thereby ensuring that the linear transformation of deep features can maintain low computational complexity. Experiments on two RSI datasets demonstrate the effectiveness of DCF in improving the classification performances of deep features without increasing dimensionality.