Toward Achieving Robust Low-Level and High-Level Scene Parsing

In this paper, we address the challenging task of scene segmentation. We first discuss and compare two widely used approaches to retain detailed spatial information from pre-trained convolutional context network (CNN)—“dilation” and “skip”. Then, we demonstrate that the parsing performance of “skip” network can be noticeably improved by modifying the parameterization of skip layers. Furthermore, we introduce a “dense skip” architecture to retain a rich set of low-level information from the pre-trained CNN, which is essential to improve the low-level parsing performance. Meanwhile, we propose a CCN and place it on top of pre-trained CNNs, which is used to aggregate contexts for high-level feature maps so that robust high-level parsing can be achieved. We name our segmentation network enhanced fully convolutional network (EFCN) based on its significantly enhanced structure over FCN. Extensive experimental studies justify each contribution separately. Without bells and whistles, EFCN achieves state-of-the-arts on segmentation datasets of ADE20K, Pascal Context, SUN-RGBD, and Pascal VOC 2012.