Image ordinal estimation: Classification and regression benefit each other

General multi-task learning requires multi-domain knowledges. In this paper, we present a double-task learning model that implements simultaneously classification and regression by only one domain knowledge: ordinal value. We jointly optimize category classification and score regression by two sibling loss in double-task convolutional neural network (DTCNN), in which category and score can be converted to one another. Experimental results show that DTCNN obtains better performance compared with individual task learning. Through analysis of different category partition we observe that fine category partition can further improve regression performance. In addition, we explain why DTCNN works better from three perspectives: (1) the relationship between two tasks from Bayesian decision rule, (2) the relationship between two tasks seen as coarse and fine classification, (3) neurons' activation on two tasks.