Fusion of CNN- and COSFIRE-Based Features with Application to Gender Recognition from Face Images

Convolution neural networks (CNNs) have been demonstrated to be very effective in various computer vision tasks. The main strength of such networks is that features are learned from some training data. In cases where training data is not abundant, transfer learning can be used in order to adapt features that are pre-trained from other tasks. Similarly, the COSFIRE approach is also trainable as it configures filters to be selective for features selected from training data. In this study we propose a fusion method of these two approaches and evaluate their performance on the application of gender recognition from face images. In particular, we use the pre-trained VGGFace CNN, which when used as standalone, it achieved 97.45% on the GENDER-FERET data set. With one of the proposed fusion approaches the recognition rate on the same task is improved to 98.9%, that is reducing the error rate by more than 50%. Our experiments demonstrate that COSFIRE filters can provide complementary features to CNNs, which contribute to a better performance.

[1]  George Azzopardi,et al.  Fusion of Domain-Specific and Trainable Features for Gender Recognition From Face Images , 2018, IEEE Access.

[2]  Bin Li,et al.  CNN-Based Adversarial Embedding for Image Steganography , 2019, IEEE Transactions on Information Forensics and Security.

[3]  Yoshua Bengio,et al.  Globally Trained Handwritten Word Recognizer Using Spatial Representation, Convolutional Neural Networks, and Hidden Markov Models , 1993, NIPS.

[4]  George Azzopardi,et al.  Gender Recognition from Face Images Using a Fusion of SVM Classifiers , 2016, ICIAR.

[5]  Sven Behnke,et al.  Large-scale object recognition with CUDA-accelerated hierarchical neural networks , 2009, 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems.

[6]  Claudio A. Perez,et al.  Gender Classification Based on Fusion of Different Spatial Scale Features Selected by Mutual Information From Histogram of LBP, Intensity, and Shape , 2013, IEEE Transactions on Information Forensics and Security.

[7]  Peng Wang,et al.  Temporal Pyramid Pooling-Based Convolutional Neural Network for Action Recognition , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Klaus Kofler,et al.  Performance and Scalability of GPU-Based Convolutional Neural Networks , 2010, 2010 18th Euromicro Conference on Parallel, Distributed and Network-based Processing.

[10]  Ranjit Kumar,et al.  Gender Recognition Through Face Using Deep Learning , 2018 .

[11]  Robinson Piramuthu,et al.  HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Kaizhu Huang,et al.  Robust Text Detection in Natural Scene Images , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  George Azzopardi,et al.  Trainable COSFIRE Filters for Keypoint Detection and Pattern Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[15]  Ho Joon Kim,et al.  Human Action Recognition Using a Modified Convolutional Neural Network , 2007, ISNN.

[16]  Shan Sung Liew,et al.  Gender classification: a convolutional neural network approach , 2016 .

[17]  George Azzopardi,et al.  A Shape Descriptor Based on Trainable COSFIRE Filters for the Recognition of Handwritten Digits , 2013, CAIP.

[18]  George Azzopardi,et al.  Color-blob-based COSFIRE filters for object recognition , 2017, Image Vis. Comput..

[19]  George Azzopardi,et al.  Trainable COSFIRE filters for vessel delineation with application to retinal images , 2015, Medical Image Anal..

[20]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[21]  Tong Zhang,et al.  Effective Use of Word Order for Text Categorization with Convolutional Neural Networks , 2014, NAACL.

[22]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[23]  George Azzopardi,et al.  Detection of curved lines with B-COSFIRE filters: A case study on crack delineation , 2017, CAIP.

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[26]  Peng Wang,et al.  Semantic Clustering and Convolutional Neural Network for Short Text Categorization , 2015, ACL.

[27]  Caifeng Shan,et al.  Learning local binary patterns for gender classification on real-world face images , 2012, Pattern Recognit. Lett..

[28]  George Azzopardi,et al.  Ventral-stream-like shape representation: from pixel intensity values to trainable object-selective COSFIRE models , 2014, Front. Comput. Neurosci..

[29]  Nina Narodytska,et al.  Simple Black-Box Adversarial Attacks on Deep Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Nicolai Petkov,et al.  Delineation of line patterns in images using B-COSFIRE filters , 2017, 2017 International Conference and Workshop on Bioinspired Intelligence (IWOBI).

[31]  Yu-Bin Yang,et al.  Text detection based on convolutional neural networks with spatial pyramid pooling , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[32]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Quoc V. Le,et al.  Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis , 2011, CVPR 2011.

[34]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Jugurta R. Montalvão Filho,et al.  Speech Recognition in Noisy Environments with Convolutional Neural Networks , 2015, 2015 Brazilian Conference on Intelligent Systems (BRACIS).

[36]  Yifan Gong,et al.  An analysis of convolutional neural networks for speech recognition , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  George Azzopardi,et al.  A Push-Pull CORF Model of a Simple Cell with Antiphase Inhibition Improves SNR and Contour Detection , 2014, PloS one.

[38]  Seyed-Mohsen Moosavi-Dezfooli,et al.  DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Nina Narodytska,et al.  Simple Black-Box Adversarial Perturbations for Deep Networks , 2016, ArXiv.

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  George Azzopardi,et al.  Identification of milling inserts in situ based on a versatile machine vision system , 2017 .

[42]  Tal Hassner,et al.  Age and gender classification using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[43]  Ben Taskar,et al.  Sidestepping Intractable Inference with Structured Ensemble Cascades , 2010, NIPS.

[44]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[45]  Ronan Collobert,et al.  Recurrent Convolutional Neural Networks for Scene Labeling , 2014, ICML.

[46]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[47]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[48]  Didier Stricker,et al.  Comparison of Kinect V1 and V2 Depth Images in Terms of Accuracy and Precision , 2016, ACCV Workshops.

[49]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[50]  Andrew Zisserman,et al.  Progressive search space reduction for human pose estimation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[52]  Daniel González-Jiménez,et al.  Single- and cross- database benchmarks for gender classification under unconstrained settings , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[53]  Václav Hlavác,et al.  Facial Landmark Tracking by Tree-Based Deformable Part Model Based Detector , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[54]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  George Azzopardi,et al.  Gender recognition from face images with trainable COSFIRE filters , 2016, 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[56]  Yongzhao Zhan,et al.  Learning Salient Features for Speech Emotion Recognition Using Convolutional Neural Networks , 2014, IEEE Transactions on Multimedia.

[57]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[58]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[59]  Isabelle Guyon,et al.  Design of a neural network character recognizer for a touch terminal , 1991, Pattern Recognit..

[60]  Chenyu Shi,et al.  Recognition of Architectural and Electrical Symbols by COSFIRE Filters with Inhibition , 2015, CAIP.