Multibranch Selective Kernel Networks for Hyperspectral Image Classification

Convolutional neural networks (CNNs) have demonstrated excellent performance in hyperspectral image (HSI) classification. However, tuning some critical hyperparameters of a CNN—such as the receptive field (RF) size—presents a major challenge due to the presence of features with different scales in HSIs. Contrary to the conventional design of CNNs, which fixes the RF size, it has been proven that the RF size is modulated by the stimulus and hence, depends on the scene being considered. Such a dilemma has been rarely considered in CNN design. In this letter, a new multibranch selective kernel network (MSKNet) is introduced, in which the input image is convolved using different RF sizes to create multiple branches so that the effect of each branch is adjusted by an attention mechanism according to the input contrast. As a result, our newly developed MSKNet is capable of modeling different scales. Our experimental results, conducted on three widely used HSIs, reveal that the MSKNet can outperform state-of-the-art CNNs in the context of HSI classification problems. The source code of our newly developed MSKNet is available from: https://github.com/mhaut/MSKNet-HSI