Hybrid-loss supervision for deep neural network

Abstract Multi-loss-joint-optimization has been proven to be valid in computer vision literature. However, the learned deep sub-features usually fit their disjoint constraints, which yield confrontation and spatial inconsistency among the sub-features with nonshared FC layers. In this paper, we propose a Hybrid-loss supervision (HLS) framework in order to obtain smoother and more spatially consistent features with shared FC layers. First, we analyze the shortcomings of the monitoring with single-loss in the existing framework theoretically. Then, we selected two notable loss functions (e.g., Center loss and Weighted loss) to instantiate the HLS framework by linear combination. By instantiating the framework with two standard loss functions, the network has learned more compact intra-class deep features and uniform inter-class deep features. The HLS framework can significantly boost the efficiency of existing convolution networks for both image classification task and object detection task without increasing network parameters and computational complexity. Extensive experimental results on different vision tasks demonstrate consistent improvement can be achieved across a variety of datasets (e.g., CIFAR-10/100, ImageNet-2012, PASCAL VOC and MS-COCO) and different convolutional neural network architectures.