Face recognition outside the visible spectrum

Abstract Automated face recognition (FR) is a well-studied problem, with a history of more than three decades. Facial recognition software that uses two dimensional (2D) images has advanced significantly in terms of accuracy over recent years. Despite the steps forward, face recognition is still not considered to be a solved problem for the cases of (i) difficult exposure conditions, such as during night-time, or in environments with unconstrained lighting, (ii) when operating at long standoff distances or at variable distances from the camera, and (iii) when using different camera sensors that operate in the same or different spectral bands. In practical forensic scenarios, it is often the case that investigators need to operate in difficult conditions, where face images captured in the aforementioned challenging cases need to be matched against good quality face images (gallery set) and where, grouping of the data in the context of demographic information (in terms of gender, ethnicity or other soft biometrics) may also be used in order to assist law enforcement officials, forensic investigators and security personnel in subject identification. In fact, in such practical scenarios, human recognition based solely on visible spectral images may not be feasible. Hence, an attractive solution is to develop efficient FR systems that can work in different practical scenarios. While different systems have been discussed and proposed over the years, an interesting approach is to increase the plurality of the source data, or in other words, start using data captured by camera sensors that cover a broad range of the electromagnetic spectrum that falls outside the visible band, including the ultraviolet (UV) and the infrared (IR) band. Thus, FR outside the visible spectrum is considered an area of growing research interest. This is further supported by the burgeoning market of camera sensors that continues to open new opportunities, at different spectral and spatial scales, in a large number of applications including those in security, forensics and defense. In light of these thoughts, the focus of this work is to expose readers to a plethora of FR advancements when operating outside the visible spectrum and our perspective on where this field is going. The discussion will include the latest studies in the UV (100–400 nm), active IR band (0.7–2.5 μm) and, finally, the passive IR band, or more specifically, the mid-wave IR (3–5 μm) and long-wave IR (8–14 μm) bands. The discussion will also include research work that involves same-spectral and cross-spectral matching scenarios, where, first, face images (probes) are acquired under controlled or difficult conditions as explained above, including dealing with uncooperative subjects, and adverse environments. Then, depending on the FR scenario, the acquired probe images are matched against mug shots collected in a controlled indoor environment using a conventional camera. This paper will start by briefly covering different multi-spectral FR systems and, then, discussing various related topics (ranging from data acquisition sets, up to system performance). The paper will also suggest needed new avenues of inquiry in the context of current work in multi-spectral FR system.