Super-resolution for Selfie Biometrics: Introduction and Application to Face and Iris

Biometrics research is heading towards enabling more relaxed acquisition conditions. This has effects on the quality and resolution of acquired images, severely affecting the accuracy of recognition systems if not tackled appropriately. In this chapter, we give an overview of recent research in super-resolution reconstruction applied to biometrics, with a focus on face and iris images in the visible spectrum, two prevalent modalities in selfie biometrics. After an introduction to the generic topic of super-resolution, we investigate methods adapted to cater for the particularities of these two modalities. By experiments, we show the benefits of incorporating super-resolution to improve the quality of biometric images prior to recognition.

[1]  Michael Unser,et al.  Image interpolation and resampling , 2000 .

[2]  Patrick J. Flynn,et al.  Iris Recognition Using Signal-Level Fusion of Frames From Video , 2009, IEEE Transactions on Information Forensics and Security.

[3]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[4]  Yung-Hui Li,et al.  Extending the Capture Volume of an Iris Recognition System Using Wavefront Coding and Super-Resolution , 2016, IEEE Transactions on Cybernetics.

[5]  Michal Irani,et al.  Super resolution from image sequences , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[6]  Peter Cheeseman,et al.  Super-Resolved Surface Reconstruction from Multiple Images , 1996 .

[7]  Kin-Man Lam,et al.  Face hallucination based on sparse local-pixel structure , 2014, Pattern Recognit..

[8]  Sridha Sridharan,et al.  Feature-domain super-resolution for iris recognition , 2011, 2011 18th IEEE International Conference on Image Processing.

[9]  Libor Masek,et al.  Recognition of Human Iris Patterns for Biometric Identification , 2003 .

[10]  Klamer Schutte,et al.  Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution , 2006, EURASIP J. Adv. Signal Process..

[11]  Haiqing Li,et al.  Image Super-Resolution for Mobile Iris Recognition , 2016, CCBR.

[12]  Sridha Sridharan,et al.  Super-resolution for biometrics: A comprehensive survey , 2018, Pattern Recognit..

[13]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[14]  Julian Fiérrez,et al.  Quality Measures in Biometric Systems , 2012, IEEE Security & Privacy.

[15]  Philip J. Willis,et al.  Image Interpolation by Pixel‐Level Data‐Dependent Triangulation , 2004, Comput. Graph. Forum.

[16]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Fernando Alonso-Fernandez,et al.  Quality factors affecting iris segmentation and matching , 2013, 2013 International Conference on Biometrics (ICB).

[18]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[19]  Tieniu Tan,et al.  Code-level information fusion of low-resolution iris image sequences for personal identification at a distance , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[20]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[21]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[22]  S. Sridharan,et al.  Focus-score weighted super-resolution for uncooperative iris recognition at a distance and on the move , 2010, 2010 25th International Conference of Image and Vision Computing New Zealand.

[23]  Junping Zhang,et al.  Super-resolution of human face image using canonical correlation analysis , 2010, Pattern Recognit..

[24]  Rafael C. González,et al.  Digital image processing, 3rd Edition , 2008 .

[25]  Julian Fiérrez,et al.  Multiple classifiers in biometrics. Part 2: Trends and challenges , 2018, Inf. Fusion.

[26]  Liang Lin,et al.  Attention-Aware Face Hallucination via Deep Reinforcement Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Kiran B. Raja,et al.  Smartphone based visible iris recognition using deep sparse filtering , 2015, Pattern Recognit. Lett..

[28]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Yu Hu,et al.  From Local Pixel Structure to Global Image Super-Resolution: A New Face Hallucination Framework , 2011, IEEE Transactions on Image Processing.

[30]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[31]  Anand Deshpande,et al.  Super resolution and recognition of long range captured multi-frame iris images , 2017, IET Biom..

[32]  Reuben A. Farrugia,et al.  Face Hallucination Using Linear Models of Coupled Sparse Support , 2015, IEEE Transactions on Image Processing.

[33]  Julian Fiérrez,et al.  Multiple classifiers in biometrics. part 1: Fundamentals and review , 2018, Inf. Fusion.

[34]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[35]  Shiguang Shan,et al.  Aligning Coupled Manifolds for Face Hallucination , 2009, IEEE Signal Processing Letters.

[36]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[37]  Dmitriy Vatolin,et al.  Fast video super-resolution via classification , 2008, 2008 15th IEEE International Conference on Image Processing.

[38]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[39]  Michael Elad,et al.  Fast and robust super-resolution , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[40]  Kaamran Raahemifar,et al.  A performance comparison among different super-resolution techniques , 2016, Comput. Electr. Eng..

[41]  Haidawati Nasir,et al.  Singular value decomposition based fusion for super-resolution image reconstruction , 2011, ICSIPA.

[42]  Reuben A. Farrugia,et al.  Exploring deep learning image super-resolution for iris recognition , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[43]  K. Harshika.  Image Quality Assessment for Fake Biometric Detection : Application to Iris , Fingerprint , and Face Recognition , 2017 .

[44]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[45]  H. K. Huang.  Handbook of Medical Imaging. Volumes 1-3 , 2007 .

[46]  Anil K. Jain,et al.  Biometric Recognition : An Overview , 2012 .

[47]  Reuben A. Farrugia,et al.  Eigen-patch iris super-resolution for iris recognition improvement , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[48]  Seong-Whan Lee,et al.  An Example-Based Face Hallucination Method for Single-Frame, Low-Resolution Facial Images , 2008, IEEE Transactions on Image Processing.

[49]  Shmuel Peleg,et al.  Efficient super-resolution and applications to mosaics , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[50]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[51]  Arun Ross,et al.  Information fusion in low-resolution iris videos using Principal Components Transform , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[52]  Kang Ryoung Park,et al.  Super-Resolution Method Based on Multiple Multi-Layer Perceptrons for Iris Recognition , 2009, Proceedings of the 4th International Conference on Ubiquitous Information Technologies & Applications.

[53]  Thomas B. Moeslund,et al.  Super-resolution: a comprehensive survey , 2014, Machine Vision and Applications.

[54]  Tieniu Tan,et al.  Learning Based Resolution Enhancement of Iris Images , 2003, BMVC.

[55]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[56]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[57]  Nicu Sebe,et al.  Neighborhood issue in single-frame image super-resolution , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[58]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[59]  Bernadette Dorizzi,et al.  Impact of Quality-Based Fusion Techniques for Video-Based Iris Recognition at a Distance , 2015, IEEE Transactions on Information Forensics and Security.

[60]  Marios Savvides,et al.  IRIS super-resolution via nonparametric over-complete dictionary learning , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[61]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[62]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Sridha Sridharan,et al.  Quality-Driven Super-Resolution for Less Constrained Iris Recognition at a Distance and on the Move , 2011, IEEE Transactions on Information Forensics and Security.

[64]  Rama Chellappa,et al.  Super-Resolution of Face Images Using Kernel PCA-Based Prior , 2007, IEEE Transactions on Multimedia.

[65]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[66]  Shao-Yi Chien,et al.  Eigen-patch: Position-patch based face hallucination using eigen transformation , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[67]  Michael Elad,et al.  Robust shift and add approach to superresolution , 2003, SPIE Optics + Photonics.

[68]  Sridha Sridharan,et al.  Feature-domain super-resolution framework for Gabor-based face and iris recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[70]  R. Barnard,et al.  High-resolution iris image reconstruction from low-resolution imagery , 2006, SPIE Optics + Photonics.

[71]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[72]  Sridha Sridharan,et al.  Robust mean super-resolution for less cooperative NIR iris recognition at a distance and on the move , 2010, SoICT '10.

[73]  Giovanni Ramponi,et al.  A simple edge-sensitive image interpolation filter , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[74]  Anand Deshpande,et al.  Multi-frame super-resolution for long range captured iris polar image , 2017, IET Biom..

[75]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[76]  Arun Ross,et al.  50 years of biometric research: Accomplishments, challenges, and opportunities , 2016, Pattern Recognit. Lett..

[77]  Chun Qi,et al.  Position-based face hallucination method , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[78]  G. Fahmy.  Super-Resolution Construction of IRIS Images from a Visual Low Resolution Face Video , 2007, 2007 National Radio Science Conference.

[79]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[80]  Reuben A. Farrugia,et al.  Robust face hallucination using quantization-adaptive dictionaries , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[81]  Reuben A. Farrugia,et al.  Iris Super-Resolution Using Iterative Neighbor Embedding , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[82]  Maoguo Gong,et al.  Position-Patch Based Face Hallucination Using Convex Optimization , 2011, IEEE Signal Processing Letters.