Sum-Fusion and Cascaded Interpolation for Semantic Image Segmentation

Semantic image segmentation classifies every pixel in an image into categories but it is difficult for a model to be good at extracting features of every category for segmentation. As features in a model may be excel at classifying a specific class, combining different models may yield a better throughput, but it necessitates heavy parameter tuning. We propose to compromise to combine several convolutional layers of different kernel sizes to get more detailed information. In our proposed algorithm, we preserve the original structure of fully convolution network but replace the convolution layer after the last Pooling layer with four convolution layers of different kernel sizes to extract multi-scale information and then four sets of feature maps obtained after the four layers are element-wise sum-fused to one set followed with convolution operation. We also propose to employ cascaded interpolation for deconvolution to get score maps as large as the corresponding input image. We evaluate our algorithm on SIFTFLOW dataset, and we really improve the segmentation accuracy.