CED-Net: context-aware ear detection network for unconstrained images

Personal authentication systems based on biometric have seen a strong demand mainly due to the increasing concern in various privacy and security applications. Although the use of each biometric trait is problem dependent, the human ear has been found to have enough discriminating characteristics to allow its use as a strong biometric measure. To locate an ear in a face image is a strenuous task, numerous existing approaches have achieved significant performance, but the majority of studies are based on the constrained environment. However, ear biometrics possess a great level of difficulties in the unconstrained environment, where pose, scale, occlusion, illuminations, background clutter, etc., vary to a great extent. To address the problem of ear detection in the wild, we have proposed two high-performance ear detection models: CED-Net-1 and CED-Net-2, which are fundamentally based on deep convolutional neural networks and primarily use contextual information to detect ear in the unconstrained environment. To compare the performance of proposed models, we have implemented state-of-the-art deep learning models, viz. FRCNN (faster region convolutional neural network) and SSD (single shot multibox detector) for ear detection task. To test the model’s generalization, these are evaluated on six different benchmark datasets, viz. IITD, IITK, USTB-DB3, UND-E, UND-J2 and UBEAR, and each one of the databases has different challenging images. The models are compared based on performance measure parameters such as IOU (intersection over union), accuracy, precision, recall and F1-score. It is observed that our proposed models CED-Net-1 and CED-Net-2 outperformed the FRCNN and SSD at higher values of IOUs. An accuracy of 99% is achieved at IOU 0.5 on majority of the databases. This performance signifies the importance and effectiveness of the models and indicates that the models are resilient to environmental conditions.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Bernt Schiele,et al.  What Makes for Effective Detection Proposals? , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Peter Peer,et al.  Ear recognition: More than a survey , 2016, Neurocomputing.

[4]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[5]  Peter Peer,et al.  Convolutional encoder-decoder networks for pixel-wise ear detection and segmentation , 2018, IET Biom..

[6]  Phalguni Gupta,et al.  Iris Recognition Using Discrete Cosine Transform and Relational Measures , 2015, CAIP.

[7]  Xudong Jiang,et al.  Semantic Correlation Promoted Shape-Variant Context for Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yuning Jiang,et al.  UnitBox: An Advanced Object Detection Network , 2016, ACM Multimedia.

[10]  Shijian Lu,et al.  CAD-Net: A Context-Aware Detection Network for Objects in Remote Sensing Imagery , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Hazim Kemal Ekenel,et al.  Domain Adaptation for Ear Recognition Using Deep Convolutional Neural Networks , 2017, IET Biom..

[12]  Rolando González-José,et al.  Automatic ear detection and feature extraction using Geometric Morphometrics and convolutional neural networks , 2017, IET Biom..

[13]  Anil K. Jain,et al.  Fingerprint Recognition of Young Children , 2017, IEEE Transactions on Information Forensics and Security.

[14]  Phalguni Gupta,et al.  Ear localization using hierarchical clustering , 2009, Defense + Commercial Sensing.

[15]  Yong Zhang,et al.  Context-Aware Feature and Label Fusion for Facial Action Unit Intensity Estimation With Partially Labeled Data , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Alex X. Liu,et al.  Behavior Based Human Authentication on Touch Screen Devices Using Gestures and Signatures , 2017, IEEE Transactions on Mobile Computing.

[18]  Ranjeet Ranjan Jha,et al.  IPSegNet : Deep Convolutional Neural Network Based Segmentation Framework for Iris and Pupil , 2017, 2017 13th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[19]  Stefano Soatto,et al.  Unsupervised Moving Object Detection via Contextual Information Separation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[21]  Gaurav Jaswal,et al.  Selection of optimized features for fusion of palm print and finger knuckle‐based person authentication , 2020, Expert Syst. J. Knowl. Eng..

[22]  Wei Jia,et al.  Palmprint Recognition Based on Complete Direction Representation , 2017, IEEE Transactions on Image Processing.

[23]  Gaurav Jaswal,et al.  Deformable multi-scale scheme for biometric personal identification , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[24]  Gaurav Jaswal,et al.  DeepKnuckle: revealing the human identity , 2017, Multimedia Tools and Applications.

[25]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Chenye Wu,et al.  Automated human identification using ear imaging , 2012, Pattern Recognit..

[27]  S. Ramachandran,et al.  Entropy-cum-Hough-transform-based ear detection using ellipsoid particle swarm optimization , 2015, Machine Vision and Applications.

[28]  Sambit Bakshi,et al.  Unconstrained ear detection using ensemble‐based convolutional neural network model , 2020, Concurr. Comput. Pract. Exp..

[29]  Shuangquan Wang,et al.  Continuous Authentication With Touch Behavioral Biometrics and Voice on Wearable Glasses , 2017, IEEE Transactions on Human-Machine Systems.

[30]  Han Wang,et al.  AcFR: Active Face Recognition Using Convolutional Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Ranjeet Ranjan Jha,et al.  UBSegNet: Unified Biometric Region of Interest Segmentation Network , 2017, 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR).

[32]  Shaogang Gong,et al.  Instance-Guided Context Rendering for Cross-Domain Person Re-Identification , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[33]  Mary Ann F. Harrison,et al.  Fast learning ear detection for real-time surveillance , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[34]  Phalguni Gupta,et al.  Multimodal Personal Authentication System Fusing Palmprint and Knuckleprint , 2013, ICIC.

[35]  Yi Zhang,et al.  Ear Detection under Uncontrolled Conditions with Multiple Scale Faster Region-Based Convolutional Neural Networks , 2017, Symmetry.

[36]  Madasu Hanmandlu,et al.  Automatic Ear Detection for Online Biometric Applications , 2011, 2011 Third National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics.

[37]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  S. Ramachandran,et al.  Entropy based Binary Particle Swarm Optimization and classification for ear detection , 2014, Eng. Appl. Artif. Intell..

[39]  Larry S. Davis,et al.  SSH: Single Stage Headless Face Detector , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[40]  Song Han,et al.  Gait-Based Continuous Authentication Using Multimodal Learning , 2017, 2017 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE).

[41]  Zhendong Wu,et al.  Damaged fingerprint classification by Deep Learning with fuzzy feature points , 2016, 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[42]  Muthukumar Arunachalam,et al.  An efficient ear recognition system using DWT & BLPOC , 2017, 2017 International Conference on Inventive Communication and Computational Technologies (ICICCT).

[43]  Christoph Busch,et al.  Robust localization of ears by feature level fusion and context information , 2013, 2013 International Conference on Biometrics (ICB).

[44]  N. K. A. Wahab,et al.  HEARD: An automatic human EAR detection technique , 2012, 2012 International Conference on Engineering and Technology (ICET).

[45]  Ivan Laptev,et al.  ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization , 2016, ECCV.

[46]  Haibo Li,et al.  Human Ear Localization: A Template-Based Approach , 2015 .

[47]  Ke Zhang,et al.  IMS-SSH: multiscale face detection method in unconstrained settings , 2019, J. Electronic Imaging.

[48]  Ping Yan,et al.  Biometric Recognition Using 3D Ear Shape , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Hugo Proença,et al.  UBEAR: A dataset of ear images captured on-the-move in uncontrolled conditions , 2011, 2011 IEEE Workshop on Computational Intelligence in Biometrics and Identity Management (CIBIM).

[50]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Alexander Wong,et al.  Fast YOLO: A Fast You Only Look Once System for Real-time Embedded Object Detection in Video , 2017, ArXiv.

[52]  Phalguni Gupta,et al.  An efficient ear localization technique , 2012, Image Vis. Comput..

[53]  Mohammed Bennamoun,et al.  Empowering Simple Binary Classifiers for Image Set Based Face Recognition , 2017, International Journal of Computer Vision.

[54]  Gaurav Jaswal,et al.  PixISegNet: pixel-level iris segmentation network using convolutional encoder-decoder with stacked hourglass bottleneck , 2020, IET Biom..

[55]  Yuanlong Yu,et al.  Context-Aware Spatio-Recurrent Curvilinear Structure Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).