Feature detection on 3D face surfaces for pose normalisation and recognition

This paper presents a SIFT algorithm adapted for 3D surfaces (called meshSIFT) and its applications to 3D face pose normalisation and recognition. The algorithm allows reliable detection of scale space extrema as local feature locations. The scale space contains the mean curvature in each vertex on different smoothed versions of the input mesh. The meshSIFT algorithm then describes the neighbourhood of every scale space extremum in a feature vector consisting of concatenated histograms of shape indices and slant angles. The feature vectors are reliably matched by comparing the angle in feature space. Using RANSAC, the best rigid transformation can be estimated based on the matched features leading to 84% correct pose normalisation of 3D faces from the Bosphorus database. Matches are mostly found between two face surfaces of the same person, allowing the algorithm to be used for 3D face recognition. Simply counting the number of matches allows 93.7% correct identification for face surfaces in the Bosphorus database and 97.7% when only frontal images are considered. In the verification scenario, we obtain an equal error rate of 15.0% to 5.1% (depending on the investigated face surfaces). These results outperform most other algorithms found in literature.