Facial marks for improving face recognition

Abstract Recent studies have shown that the use of soft biometrics (e.g. gender, ethnicity, facial marks) as supplementary information in face images, can increase the accuracy of the recognition process of individuals. Facial marks (e.g. moles, freckles, warts) have shown to be useful, particularly, in this regard. In this paper we propose a new method for combining existing face recognition systems with the information obtained from facial marks in order to improve their performance. We first introduce an algorithm for automatically detecting facial marks, which are then represented using Histograms of Oriented Gradients (HoG), and are matched taking into account their position in the face image. Extensive experiments are conducted in order to show the effectiveness of the proposed facial mark detection algorithm, and to corroborate the benefits of using the information of facial marks on top of traditional face recognition systems. Due to the lack of proper public benchmarks to validate facial mark detection, we also present and make available a dataset with manual annotations for this purpose.