Studying Very Low Resolution Recognition Using Deep Networks

Visual recognition research often assumes a sufficient resolution of the region of interest (ROI). That is usually violated in practice, inspiring us to explore the Very Low Resolution Recognition (VLRR) problem. Typically, the ROI in a VLRR problem can be smaller than 16 16 pixels, and is challenging to be recognized even by human experts. We attempt to solve the VLRR problem using deep learning methods. Taking advantage of techniques primarily in super resolution, domain adaptation and robust regression, we formulate a dedicated deep learning method and demonstrate how these techniques are incorporated step by step. Any extra complexity, when introduced, is fully justified by both analysis and simulation results. The resulting Robust Partially Coupled Networks achieves feature enhancement and recognition simultaneously. It allows for both the flexibility to combat the LR-HR domain mismatch, and the robustness to outliers. Finally, the effectiveness of the proposed models is evaluated on three different VLRR tasks, including face identification, digit recognition and font recognition, all of which obtain very impressive performances.

[1]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[2]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[3]  Badong Chen,et al.  Efficient and robust deep learning with Correntropy-induced loss function , 2015, Neural Computing and Applications.

[4]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[5]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Shiguang Shan,et al.  Deeply Coupled Auto-encoder Networks for Cross-view Classification , 2014, ArXiv.

[7]  Rama Chellappa,et al.  Synthesis-based Robust Low Resolution Face Recognition , 2017, ArXiv.

[8]  P. J. Huber.  Robust Estimation of a Location Parameter , 1964 .

[9]  Pablo H. Hennings-Yeomans,et al.  Simultaneous super-resolution and feature extraction for recognition of low-resolution faces , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Andrew Zisserman,et al.  Deep Features for Text Spotting , 2014, ECCV.

[11]  Pascal Vincent,et al.  The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training , 2009, AISTATS.

[12]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[13]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[14]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Tony X. Han,et al.  Large-Scale Visual Font Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[17]  Yoshua Bengio,et al.  Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach , 2011, ICML.

[18]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, BTAS.

[19]  Mislav Grgic,et al.  SCface – surveillance cameras face database , 2011, Multimedia Tools and Applications.

[20]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[22]  Antonio Torralba,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 80 Million Tiny Images: a Large Dataset for Non-parametric Object and Scene Recognition , 2022 .

[23]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Jianmin Zhang,et al.  Robust Deep Network with Maximum Correntropy Criterion for Seizure Detection , 2014, BioMed research international.

[25]  Harry Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[29]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[30]  Terrance E. Boult,et al.  Large scale unconstrained open set face database , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[31]  Thomas S. Huang,et al.  Learning Super-Resolution Jointly From External and Internal Examples , 2015, IEEE Transactions on Image Processing.

[32]  Takeo Kanade,et al.  Limits on Super-Resolution and How to Break Them , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Thomas S. Huang,et al.  Self-tuned deep super resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[34]  Bruce A. Draper,et al.  A meta-analysis of face recognition covariates , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[35]  Dahua Lin,et al.  Coupled space learning of image style transformation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[36]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[37]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[38]  Julien Mairal,et al.  Convex optimization with sparsity-inducing norms , 2011 .

[39]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[40]  Thomas S. Huang,et al.  DeepFont: Identify Your Font from An Image , 2015, ACM Multimedia.

[41]  Paul A. Viola,et al.  Text recognition of low-resolution document images , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[42]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[43]  Thomas S. Huang,et al.  Real-World Font Recognition Using Deep Network and Domain Adaptation , 2015, ICLR.

[44]  Jinhui Tang,et al.  Weakly-Shared Deep Transfer Networks for Heterogeneous-Domain Knowledge Propagation , 2015, ACM Multimedia.