Deep Ear Recognition Pipeline

Ear recognition has seen multiple improvements in recent years and still remains very active today. However, it has been approached from recognition and detection perspective separately. Furthermore, deep-learning-based approaches that are popular in other domains have seen limited use in ear recognition and even more so in ear detection. Moreover, to obtain a usable recognition system a unified pipeline is needed. The input in such system should be plain images of subjects and the output identities based only on ear biometrics. We conduct separate analysis through detection and identification experiments on the challenging dataset and, using the best approaches, present a novel, unified pipeline. The pipeline is based on convolutional neural networks (CNN) and presents, to the best of our knowledge, the first CNN-based ear recognition pipeline. The pipeline incorporates both, the detection of ears on arbitrary images of people, as well as recognition on these segmented ear regions. The experiments show that the presented system is a state-of-the-art system and, thus, a good foundation for future real-word ear recognition systems.

[1]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[2]  Peter Peer,et al.  Convolutional encoder-decoder networks for pixel-wise ear detection and segmentation , 2018, IET Biom..

[3]  Yi Zhang,et al.  Ear verification under uncontrolled conditions with convolutional neural networks , 2018, IET Biom..

[4]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[5]  Dong Zhou,et al.  Translation techniques in cross-language information retrieval , 2012, CSUR.

[6]  N. K. A. Wahab,et al.  HEARD: An automatic human EAR detection technique , 2012, 2012 International Conference on Engineering and Technology (ICET).

[7]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Mark S. Nixon,et al.  Robust log-Gabor filter for ear biometrics , 2008, 2008 19th International Conference on Pattern Recognition.

[10]  Peter Peer,et al.  Ear recognition: More than a survey , 2016, Neurocomputing.

[11]  J. Crowley,et al.  Estimating Face orientation from Robust Detection of Salient Facial Structures , 2004 .

[12]  Ahmed Bouridane,et al.  An automated ear identification system using Gabor filter responses , 2015, 2015 IEEE 13th International New Circuits and Systems Conference (NEWCAS).

[13]  Muhammad Shoaib,et al.  A human ear recognition method using nonlinear curvelet feature subspace , 2014, Int. J. Comput. Math..

[14]  Maurício Pamplona Segundo,et al.  Employing Fusion of Learned and Handcrafted Features for Unconstrained Ear Recognition , 2017, IET Biom..

[15]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[16]  Mary Ann F. Harrison,et al.  Fast learning ear detection for real-time surveillance , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[17]  Blaz Meden,et al.  Covariate analysis of descriptor-based ear recognition techniques , 2017, 2017 International Conference and Workshop on Bioinspired Intelligence (IWOBI).

[18]  H. Proenca,et al.  The NICE.I: Noisy Iris Challenge Evaluation - Part I , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[19]  Hazim Kemal Ekenel,et al.  Domain Adaptation for Ear Recognition Using Deep Convolutional Neural Networks , 2017, IET Biom..

[20]  Matti Pietikäinen,et al.  Computer Vision Using Local Binary Patterns , 2011, Computational Imaging and Vision.

[21]  Jiang Chen,et al.  A robust algorithm for ear recognition under partial occlusion , 2013, Proceedings of the 32nd Chinese Control Conference.

[22]  Xudong Jiang,et al.  Deep Coupled ResNet for Low-Resolution Face Recognition , 2018, IEEE Signal Processing Letters.

[23]  Peter Peer,et al.  Training Convolutional Neural Networks with Limited Training Data for Ear Recognition in the Wild , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[24]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Alice Caplier,et al.  Face Recognition with Patterns of Oriented Edge Magnitudes , 2010, ECCV.

[26]  Mark S. Nixon,et al.  Toward Unconstrained Ear Recognition From Two-Dimensional Images , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[27]  Bourouba Houcine,et al.  Ear recognition based on Multi-bags-of-features histogram , 2015, 2015 3rd International Conference on Control, Engineering & Information Technology (CEIT).

[28]  Lina J. Karam,et al.  Unconstrained ear recognition using deep neural networks , 2018, IET Biom..

[29]  Christoph Busch,et al.  Ear biometrics: a survey of detection, feature extraction and recognition methods , 2012, IET Biom..

[30]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[31]  T. Dillon,et al.  Electricity price short-term forecasting using artificial neural networks , 1999 .

[32]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[33]  Yong Du,et al.  Learning pairwise SVM on hierarchical deep features for ear recognition , 2018, IET Biom..

[34]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[35]  Zhengguang Xu,et al.  Ear recognition using a new local matching approach , 2008, 2008 15th IEEE International Conference on Image Processing.

[36]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[37]  S. Ramachandran,et al.  Entropy-cum-Hough-transform-based ear detection using ellipsoid particle swarm optimization , 2015, Machine Vision and Applications.

[38]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[39]  S. Prakash,et al.  Ear Localization from Side Face Images using Distance Transform and Template Matching , 2008, 2008 First Workshops on Image Processing Theory, Tools and Applications.

[40]  Christoph Busch,et al.  A comparative study on texture and surface descriptors for ear biometrics , 2014, 2014 International Carnahan Conference on Security Technology (ICCST).

[41]  Mark S. Nixon,et al.  On Shape-Mediated Enrolment in Ear Biometrics , 2007, ISVC.

[42]  Ali Borji,et al.  Analysis of Hand Segmentation in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[43]  Vitomir Struc,et al.  Adaptation of SIFT Features for Robust Face Recognition , 2010, ICIAR.

[44]  Naser Damer,et al.  Ear Recognition Using Multi-Scale Histogram of Oriented Gradients , 2012, 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[45]  Stefanos Zafeiriou,et al.  The unconstrained ear recognition challenge , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[46]  Tianfu Wang,et al.  Dense deconvolution net: Multi path fusion and dense deconvolution for high resolution skin lesion segmentation , 2018, Technology and health care : official journal of the European Society for Engineering and Medicine.

[47]  Jingyu Hou,et al.  A neural network based human identification framework using ear images , 2010, TENCON 2010 - 2010 IEEE Region 10 Conference.

[48]  Phalguni Gupta,et al.  Localization of Ear Using Outer Helix Curve of the Ear , 2007, 2007 International Conference on Computing: Theory and Applications (ICCTA'07).

[49]  Esa Rahtu,et al.  BSIF: Binarized statistical image features , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[50]  Abdelhani Boukrouche,et al.  Identity recognition based on the external shape of the human ear , 2015, 2015 International Conference on Applied Research in Computer Science and Engineering (ICAR).

[51]  Phalguni Gupta,et al.  Ear Biometrics in 2D and 3D: Localization and Recognition , 2015 .

[52]  S. Ramachandran,et al.  Entropy based Binary Particle Swarm Optimization and classification for ear detection , 2014, Eng. Appl. Artif. Intell..

[53]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[54]  Amitava Chatterjee,et al.  Robust multimodal multivariate ear recognition using kernel based simultaneous sparse representation , 2017, Eng. Appl. Artif. Intell..

[55]  Guodong Guo,et al.  On Applicability of Tunable Filter Bank Based Feature for Ear Biometrics: A Study from Constrained to Unconstrained , 2017, Journal of Medical Systems.

[56]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling , 2015, CVPR 2015.

[57]  Christoph Busch,et al.  Robust localization of ears by feature level fusion and context information , 2013, 2013 International Conference on Biometrics (ICB).

[58]  Karim Faez,et al.  A New Segmentation Approach for Ear Recognition , 2008, ACIVS.

[59]  Phalguni Gupta,et al.  An efficient ear localization technique , 2012, Image Vis. Comput..

[60]  Mohammed Bennamoun,et al.  Fast and Fully Automatic Ear Detection Using Cascaded AdaBoost , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[61]  M. A. Carreira-Perpinan,et al.  Compression neural networks for feature extraction: Application to human recognition from ear images , 1995 .

[62]  Liang Tian,et al.  Ear recognition based on deep convolutional network , 2016, 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[63]  Takashi Yahagi,et al.  Ear photo recognition using scale invariant keypoints , 2006, Computational Intelligence.

[64]  Yong Du,et al.  Learning pairwise SVM on deep features for ear recognition , 2017, 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS).

[65]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[66]  Surya Prakash,et al.  Ear Biometrics in 2D and 3D , 2015 .

[67]  Esa Rahtu,et al.  Rotation invariant local phase quantization for blur insensitive texture analysis , 2008, 2008 19th International Conference on Pattern Recognition.

[68]  Chenye Wu,et al.  Automated human identification using ear imaging , 2012, Pattern Recognit..

[69]  Phalguni Gupta,et al.  An efficient ear recognition technique invariant to illumination and pose , 2013, Telecommun. Syst..

[70]  Mark S. Nixon,et al.  A novel ray analogy for enrolment of ear biometrics , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[71]  Bhabani Shankar Prasad Mishra,et al.  An Automated Ear Localization Technique Based on Modified Hausdorff Distance , 2016, CVIP.

[72]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[73]  Tianfu Wang,et al.  Skin Lesion Segmentation via Deep RefineNet , 2017, DLMIA/ML-CDS@MICCAI.

[74]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[75]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[76]  Ajay Kumar,et al.  Reliable ear identification using 2-D quadrature filters , 2012, Pattern Recognit. Lett..

[77]  Peter Peer,et al.  Ear biometric database in the wild , 2015, 2015 4th International Work Conference on Bioinspired Intelligence (IWOBI).

[78]  Abdelhani Boukrouche,et al.  Ear description and recognition using ELBP and wavelets , 2015, 2015 International Conference on Applied Research in Computer Science and Engineering (ICAR).

[79]  Arun Ross,et al.  A survey on ear biometrics , 2013, CSUR.

[80]  Arun Ross,et al.  2D ear classification based on unsupervised clustering , 2014, IEEE International Joint Conference on Biometrics.

[81]  Phalguni Gupta,et al.  A Skin-Color and Template Based Technique for Automatic Ear Detection , 2009, 2009 Seventh International Conference on Advances in Pattern Recognition.

[82]  Phalguni Gupta,et al.  Connected component based technique for automatic ear detection , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[83]  Peter Peer,et al.  Evaluation and analysis of ear recognition models: performance, complexity and resource requirements , 2020, Neural Computing and Applications.