Saliency detection using a deep conditional random field network

Abstract Saliency detection has made remarkable progress along with the development of deep learning. While how to integrate the low-level intrinsic context with high-level semantic information to keep the object boundary sharp and restrain the background noise is still a challenging problem. Many attempts on network structures and refinement strategies have been explored, such as using Conditional Random Field (CRF) to improve the accuracy of saliency map, but it is independent from the deep network and cannot be trained end-to-end. To tackle this issue, we propose a novel Deep Conditional Random Field network (DCRF) to take both deep feature and neighbor information into consideration. First, Multi-scale Feature Extraction Module (MFEM) is adopted to capture the low level texture and high level semantic features, multi-stacks of deconvolution layers are employed to improve the spatial resolution of deep layers. Then we employ Backward Optimization Module (BOM) to guide shallower layers by high-level location and shape information derived from deeper layers, which intrinsically enhance the representational capacity of low-level features. Finally, a Deep Conditional Random Field Module (DCRFM) with unary and pairwise potentials is designed to concentrate on spatial neighbor relations to obtain a compact and uniformed saliency map. Extensive experimental results on 5 datasets in terms of 6 evaluation metrics demonstrate that the proposed method achieves state-of-the-art performance.