A DNN-based semantic segmentation for detecting weed and crop

Abstract Weed control is a global issue, and has attracted great attention in recent years. Deploying autonomous robots for weed removal has great potential in terms of constructing environment-friendly agriculture, and saving manpower. In this paper, we propose a weed/crop segmentation network that provides better performance for precisely recognizing the weed with arbitrary shape in complex environment condition, and offers great support for autonomous robots to successfully reduce the density of weed. Our deep neural network (DNN)-based segmentation model obtains persistent improvements by integrating four additional components. i) Hybrid dilated convolution and DropBlock are introduced into the classification backbone network, where the hybrid dilated convolution enlarges the receptive field, while DropBlock regularizes the weight parameters to learn robust features by random drops contiguous regions. ii) A universal function approximation block is added to the front-end of the backbone network, which adaptively converts the existing RGB-NIR bands into optimized (RGB + NIR)-based indices to increase the classification performance. iii) The bridge attention block is exploited, in order to make the network “globally” refer to the correlated region, regardless of the distance for capturing the rich long-range contextual information. iv) The spatial pyramid refinement block is inserted to fuse multi-scale feature maps with different size of receptive fields to provide the precise localization of segmentation result, by maintaining the consistency of feature maps. We evaluate our network performance on two challenging Stuttgart and Bonn datasets. The state-of-the-art performance on the two datasets shows that each added component has notable potential to boost the segmentation accuracy.