Heterogenous output regression network for direct face alignment

Abstract Face alignment has gained great popularity in computer vision due to its wide-spread applications. In this paper, we propose a novel learning architecture, i.e., heterogenous output regression network (HORNet), for face alignment, which directly predicts facial landmarks from images. HORNet is based on kernel approximations and establishes a new compact multi-layer architecture. A nonlinear layer with cosine activations disentangles nonlinear relationships between representations of images and shapes of facial landmarks. A linear layer with identity activations explicitly encodes landmark correlations by low-rank learning via matrix elastic nets. HORNet is highly flexible and can work either with pre-built feature representations or with convolutional architectures for end-to-end learning. HORNet leverages the strengths of both kernel methods in modeling nonlinearities and of neural networks in structural prediction. This combination renders it effective and efficient for direct face alignment. Extensive experiments on five in-the-wild datasets show that HORNet delivers high performance and consistently exceeds state-of-the-art methods.