Multiple convolutional layers fusion framework for hyperspectral image classification

Abstract Hyperspectral images (HSIs) often contain complex structures with different scales, and thus capturing the structural information of various scales is very important for HSIs classification. The convolutional neural network can automatically extract the features from fine to coarse scales along with the increase of convolutional layers. Therefore, in this paper, we propose a novel framework called multiple convolutional layers fusion, which aims to fuse information extracted by different convolutional layers for HSIs classification. In general, the proposed framework adopts two fusion networks: side output decision fusion network (SODFN) and fully convolutional layer fusion network (FCLFN). In more detail, SODFN optimizes each convolutional layer in the network to create a side classification map. After that, the majority voting is applied on these maps to obtain the fused result. In the FCLFN, one fully convolutional layer (FCL) is first created for each convolutional layer, and then FCLs from different convolutional layers are combined to form one FCL in the last layer. Finally, the new FCL can be used to obtain the final classification map. Experimental results on three real HSIs demonstrate the superiority of the proposed method over the traditional convolutional neural network based methods and several well-known classifiers.