FERLrTc: 2D+3D facial expression recognition via low-rank tensor completion

Abstract In this paper, a 4D tensor model is firstly constructed to explore efficient structural information and correlations from multi-modal data (both 2D and 3D face data). As the dimensionality of the generated 4D tensor is high, a tensor dimensionality reduction technique is in need. Since many real-world high-order data often reside in a low dimensional subspace, Tucker decomposition as a powerful technique is utilized to capture multilinear low-rank structure and to extract useful information from the generated 4D tensor data. Our goal is to use Tucker decomposition to obtain a set of core tensors with smaller sizes and factor matrices which are projected into the 4D tensor data for classification prediction. To characterize the involved similarities of the 4D tensor, the low-rank and sparse representation is built in terms of the low-rank structure of factor matrices and the sparsity of the core tensor in the Tucker decomposition of the generated 4D tensor. A tensor completion (TC) framework is embedded to recover the missing information in the 4D tensor modeling process. Thus, a novel tensor dimensionality reduction approach for 2D+3D facial expression recognition via low-rank tensor completion (FERLrTC) is proposed to solve the factor matrices in a majorization–minimization manner by using a rank reduction strategy. Numerical experiments are conducted with a full implementation on the BU-3DFE and Bosphorus databases and synthetic data to illustrate the effectiveness of the proposed approach.