Pitfalls in studying “big data” from operational scenarios

Analyzing a larger dataset is sometimes assumed, in itself, to confer a greater degree of validity to the results of a study. In biometrics, analyzing an “operational” dataset is also sometimes assumed, in itself to confer a greater degree of validity. And so studying a large, operational biometric dataset may seem to guarantee valid results. However, a number of basic questions should be asked of any “found” big data, in order to avoid pitfalls of the data not being suitable for the desired analysis. We explore such issues using a large operational iris recognition dataset from the Canada Border Services Agency's NEXUS program, similar to the dataset analyzed in the NISTIREX VI report.