Biometrics for cybersecurity and unconstrained environments

of a dissertation at the University of Miami. Dissertation supervised by Professor Mohamed Abdel-Mottaleb. No. of pages in text. (155) Biometric identification has been a challenging topic in computer vision for the past few decades. In this thesis, we study four main challenges in biometrics: 1) secure and privacy-preserving biometric identification in untrusted public cloud servers, 2) single sample face recognition in unconstrained environments, 3) multimodal biometrics using feature-level fusion, and 4) low-resolution face recognition. In biometric identification systems, the biometric database is typically stored on a trusted server, which is also responsible for performing the identification process. However, if this server is a public cloud, maintenance of the confidentiality and integrity of sensitive data requires trustworthy solutions for its storage and processing. In the first part of our study, we present CloudID, a privacy-preserving cloud-based biometric identification solution. It links the confidential information of the users to their biometrics and stores it in an encrypted fashion. Making use of a searchable encryption technique, biometric identification is performed in the encrypted domain to make sure that the cloud provider or potential attackers do not gain access to any sensitive data or even the contents of the individual queries. The proposed approach is the first cloud-based biometric identification system with a proven zero data disclosure possibility. It allows different enterprises to perform biometric identification on a single database without revealing any sensitive information. In the second part of this study, we present a fully automatic face recognition technique robust to face pose variations in unconstrained environments. The proposed method normalizes the face images for both in-plane and out-of-plane pose variations using an enhanced technique based on active appearance models (AAMs). We improve the performance of AAM fitting, not only by training it with in-thewild images and using a powerful optimization technique but also by initializing the AAM with estimates of the locations of the facial landmarks obtained by a method based on flexible mixture of parts. The proposed initialization technique results in significant improvement in AAM fitting to non-frontal poses and makes the normalization process robust, fast and reliable. Owing to the proper alignment of the face images, made possible by this approach, we can use local feature descriptors, such as Histograms of Oriented Gradients (HOG), which makes the system robust against illumination variations. In order to improve the discriminating information content of the feature vectors, we also extract Gabor features from the normalized face images and fuse them with HOG features using Canonical Correlation Analysis (CCA). The proposed face recognition system is capable of recognizing faces from non-frontal views and under different illumination conditions using only a single gallery sample for each subject. This is important because of its potential applications in real life applications such as video surveillance. In the third part of this study, we propose a real-time feature level fusion technique for multimodal biometric systems. The goal of feature fusion for recognition is to combine relevant information from two or more feature vectors into a single one with more discriminative power than any of the input feature vectors. In pattern recognition problems, we are also interested in separating the classes. In this study, we present Discriminant Correlation Analysis (DCA), a feature level fusion technique that incorporates the class associations into the correlation analysis of the feature sets. DCA performs an effective feature fusion by maximizing the pair-wise correlations across the two feature sets, and at the same time, eliminating the between-class correlations and restricting the correlations to be within the classes. The proposed method can be used in pattern recognition applications for fusing features extracted from multiple modalities or combining different feature vectors extracted from a single modality. DCA has a very low computational complexity and it can be employed in real-time applications. Multiple sets of experiments performed on various biometric databases, and using different feature extraction techniques, show the effectiveness of the proposed method, which outperforms other state-of-the-art approaches. In the fourth and last part of this thesis, we propose a novel real-time approach for matching Low Resolution (LR) probe face images with High Resolution (HR) gallery face images with an application to surveillance systems. The proposed method is based on DCA. It projects the LR and HR feature vectors in a common domain in which not only the LR and HR feature vectors have the same length, but also the correlation between corresponding features in LR and HR domain is maximized. In addition, the process of calculating the projection matrices considers the class structure of the data and it aims to separate the classes in the new domain, which is very beneficial from the recognition perspective. It is worth mentioning that the proposed method has a very low computational complexity and it can be employed for processing several faces that appear in a crowded image in real-time. Experiments performed on low-resolution surveillance images verify the effectiveness of our proposed method in comparison with the state-of-the-art LR face recognition techniques. Dedicated to Claire, Badoum, April, Teatop, and Tasha