GenLR-Net: Deep Framework for Very Low Resolution Face and Object Recognition with Generalization to Unseen Categories

Matching very low resolution images of faces and objects with high resolution images in the database has important applications in surveillance scenarios, street-to-shop matching for general objects, etc. Matching across huge resolution difference along with variations in illumination, view-point, etc. makes the problem quite challenging. The problem becomes even more difficult if the testing objects have not been seen during training. In this work, we propose a novel deep convolutional neural network architecture to address these problems. We systematically introduce different kinds of constraints at different stages of the architecture so that the approach can recognize low resolution images as well as generalize well to images of unseen categories. The reason behind each additional step along with its effect on the overall performance is thoroughly analyzed. Extensive experiments are conducted on two face and object datasets which justifies the effectiveness of the proposed approach for handling these real-life challenging scenarios.

[1]  Bernt Schiele,et al.  Zero-Shot Learning — The Good, the Bad and the Ugly , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Subhransu Maji,et al.  Adapting Models to Signal Degradation using Distillation , 2017, BMVC.

[3]  Svetlana Lazebnik,et al.  Where to Buy It: Matching Street Clothing Photos in Online Shops , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[5]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Xiaogang Wang,et al.  DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Sivaram Prasad Mudunuri,et al.  Low Resolution Face Recognition Across Variations in Pose and Illumination , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Qingmin Liao,et al.  Discriminative Multidimensional Scaling for Low-Resolution Face Recognition , 2018, IEEE Signal Processing Letters.

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Qijun Zhao,et al.  Towards resolution invariant face recognition in uncontrolled scenarios , 2016, 2016 International Conference on Biometrics (ICB).

[12]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[13]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[14]  Qiang Chen,et al.  Cross-Domain Image Retrieval with a Dual Attribute-Aware Ranking Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Carlos D. Castillo,et al.  Frontal to profile face verification in the wild , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[18]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  Lei Zhang,et al.  Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[21]  Ke Chen,et al.  Convolutional Low-Resolution Fine-Grained Classification , 2017, Pattern Recognit. Lett..

[22]  Q. M. Jonathan Wu,et al.  Low-resolution face recognition: a review , 2013, The Visual Computer.

[23]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[24]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Gregory Shakhnarovich,et al.  Examining the Impact of Blur on Recognition by Convolutional Networks , 2016, ArXiv.

[26]  Frédéric Jurie,et al.  Learning Visual Similarity Measures for Comparing Never Seen Objects , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Gustavo K. Rohde,et al.  Transport-based single frame super resolution of very low resolution face images , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Kate Saenko,et al.  Fine-to-coarse knowledge transfer for low-res image classification , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[29]  Trevor Darrell,et al.  Simultaneous Deep Transfer Across Domains and Tasks , 2015, ICCV.

[30]  Gang Wang,et al.  A Siamese Long Short-Term Memory Architecture for Human Re-identification , 2016, ECCV.

[31]  Reuben A. Farrugia,et al.  Face Hallucination Using Linear Models of Coupled Sparse Support , 2015, IEEE Transactions on Image Processing.

[32]  Thomas S. Huang,et al.  Studying Very Low Resolution Recognition Using Deep Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Rama Chellappa,et al.  Synthesis-based Robust Low Resolution Face Recognition , 2017, ArXiv.

[34]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Xiaoou Tang,et al.  Deep Cascaded Bi-Network for Face Hallucination , 2016, ECCV.

[36]  Sivaram Prasad Mudunuri,et al.  Dictionary Alignment for Low-Resolution and Heterogeneous Face Recognition , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[37]  Xin Yu,et al.  Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).