A BioSecure ( DS 2 ) Report on the Technological Evaluation of Score-level Quality-dependent and Cost-sensitive Multimodal Biometric Performance

This report summarizes the result of the BioSEcure DS2 (Desktop) evaluation campaign. This campaign aims at evaluating multimodal fusion algorithms involving face, fingerprint and iris biometrics for person authentication, targeting at the application of physical access control in a medium-sized establishment with some 500 persons. While multimodal biometrics is a well investigated subject in the literature, there exists no benchmark results on which basis fusion algorithms can be compared. Working towards this goal, we designed two sets of experiments: quality-dependent and cost-sensitive evaluations. The quality-dependent evaluation aims at evaluating how well fusion algorithms can perform under changing quality of raw biometric images principally due to change of devices. The cost-sensitive evaluation, on the other hand, aims at how well a fusion algorithm can perform given restricted computation and in the presence of software and hardware failures, resulting in errors such as failure to acquire and failure to match. Since multiple capturing devices are available, a fusion algorithm should be able to handle this non-ideal but nevertheless realistic scenario. It is on this ground that this evaluation is proposed. In both evaluations, a fusion algorithm is supplied with scores from each biometric comparison subsystems as well as the quality measures of both the template and the quality measures. The evaluation campaign is very encouraging, receiving 15 fusion systems. To the best of our knowledge, the BioSecure DS2 evaluation campaign is the first attempt to benchmark both quality-dependent and cost-sensitive fusion algorithms. Our evaluation suggests that while using all the available biometric sensors can definitely increase the fusion performance, one has to tradeoff with the increased cost in terms of acquisition time, computation time and the physical cost of hardware maintenance. A promising solution which does not increase this composite factors of cost is dynamic fusion, as demonstrated in our experiments. In the presence of changing image quality which may due to change of acquisition devices and/or device capturing configurations, we observe that the top performing fusion algorithms are those that exploit the automatically derived quality measures in order to recover the most probable biometric device from which the associated biometric data was scanned.