Dual Directed Capsule Network for Very Low Resolution Image Recognition

Very low resolution (VLR) image recognition corresponds to classifying images with resolution 16x16 or less. Though it has widespread applicability when objects are captured at a very large stand-off distance (e.g. surveillance scenario) or from wide angle mobile cameras, it has received limited attention. This research presents a novel Dual Directed Capsule Network model, termed as DirectCapsNet, for addressing VLR digit and face recognition. The proposed architecture utilizes a combination of capsule and convolutional layers for learning an effective VLR recognition model. The architecture also incorporates two novel loss functions: (i) the proposed HR-anchor loss and (ii) the proposed targeted reconstruction loss, in order to overcome the challenges of limited information content in VLR images. The proposed losses use high resolution images as auxiliary data during training to "direct" discriminative feature learning. Multiple experiments for VLR digit classification and VLR face recognition are performed along with comparisons with state-of-the-art algorithms. The proposed DirectCapsNet consistently showcases state-of-the-art results; for example, on the UCCS face database, it shows over 95% face recognition accuracy when 16x16 images are matched with 80x80 images.

[1]  Richa Singh,et al.  Detecting and Mitigating Adversarial Perturbations for Robust Face Recognition , 2019, International Journal of Computer Vision.

[2]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[3]  Kin-Man Lam,et al.  Simultaneous Hallucination and Recognition of Low-Resolution Faces Based on Singular Value Decomposition , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Geoffrey E. Hinton,et al.  Transforming Auto-Encoders , 2011, ICANN.

[5]  Chen Xu,et al.  MS-CapsNet: A Novel Multi-Scale Capsule Network , 2018, IEEE Signal Processing Letters.

[6]  Rama Chellappa,et al.  Face Verification Across Age Progression , 2006, IEEE Trans. Image Process..

[7]  Rama Chellappa,et al.  Recognizing Disguised Faces in the Wild , 2018, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[8]  Terrance E. Boult,et al.  Large scale unconstrained open set face database , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[9]  Himanshu S. Bhatt,et al.  Submitted to Ieee Transactions on Image Processing 1 Improving Cross-resolution Face Matching Using Ensemble Based Co-transfer Learning , 2022 .

[10]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[11]  Q. M. Jonathan Wu,et al.  Low-resolution face recognition: a review , 2013, The Visual Computer.

[12]  Timothy Hospedales,et al.  A survey on heterogeneous face recognition: Sketch, infra-red, 3D and low-resolution , 2014, Image Vis. Comput..

[13]  Thomas S. Huang,et al.  Studying Very Low Resolution Recognition Using Deep Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Q. Mcnemar Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[15]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[16]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[17]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[19]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[20]  Richa Singh,et al.  Identity Aware Synthesis for Cross Resolution Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Premkumar Natarajan,et al.  CapsuleGAN: Generative Adversarial Capsule Network , 2018, ECCV Workshops.

[22]  Sivaram Prasad Mudunuri,et al.  Low Resolution Face Recognition Across Variations in Pose and Illumination , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Patrick J. Flynn,et al.  On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques , 2018, IEEE Transactions on Information Forensics and Security.

[24]  Jiang Li,et al.  Seagrass Detection in Coastal Water Through Deep Capsule Networks , 2018, PRCV.

[25]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[26]  Konstantinos N. Plataniotis,et al.  Brain Tumor Type Classification via Capsule Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[27]  Chao Dong,et al.  Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Shiming Ge,et al.  Low-Resolution Face Recognition in the Wild via Selective Knowledge Distillation , 2018, IEEE Transactions on Image Processing.

[29]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).