Component-based feature extraction and representation schemes for vehicle make and model recognition

Abstract This paper presents a novel feature representation and recognition scheme for vehicle make and model recognition (VMMR) from the frontal image of the vehicle. In general, some domain knowledge is introduced and further exploited to accomplish this task. The modular components in the frontal appearance of vehicles present distinct visual characteristics, and the varying discrimination ability of them made the main discriminant region changes when vehicles compared at inter- or intra-brand level. Inspired by the peculiar vehicle properties, this paper focuses on the localized visual characteristics in the discriminant subregions, and make the representation of the frontal vehicle in a multi-scale spatial manner. This representation scheme encodes local spatial information and component-specific characteristics to the feature descriptors, which enhances the discrimination ability of the feature representation and mitigates the multiplicity problem of VMMR to some extent. Extensive experiments on a large-scale vehicle image dataset have shown the efficiency of the methods in this work.

[1]  Meena Abdelmaseeh,et al.  Car Make and Model recognition combining global and local cues , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[2]  Bir Bhanu,et al.  Dynamic Bayesian Networks for Vehicle Classification in Video , 2012, IEEE Transactions on Industrial Informatics.

[3]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Yunde Jia,et al.  Vehicle Type Classification Using a Semisupervised Convolutional Neural Network , 2015, IEEE Transactions on Intelligent Transportation Systems.

[5]  Matthew Turk,et al.  Car-Rec: A real time car recognition system , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[6]  Wei Wu,et al.  A method of vehicle classification using models and neural networks , 2001, IEEE VTS 53rd Vehicular Technology Conference, Spring 2001. Proceedings (Cat. No.01CH37202).

[7]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[8]  Jun-Wei Hsieh,et al.  Symmetrical SURF and Its Applications to Vehicle Detection and Vehicle Make and Model Recognition , 2014, IEEE Transactions on Intelligent Transportation Systems.

[9]  W. Eric L. Grimson,et al.  Edge-based rich representation for vehicle classification , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[10]  Zezhi Chen,et al.  Vehicle type categorization: A comparison of classification schemes , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[11]  Eran A. Edirisinghe,et al.  Vehicle Make and Model Recognition in CCTV footage , 2013, 2013 18th International Conference on Digital Signal Processing (DSP).

[12]  Jian-Huang Lai,et al.  Location-aware fine-grained vehicle type recognition using multi-task deep networks , 2017, Neurocomputing.

[13]  Dongjin Han,et al.  Vehicle Class Recognition from Video-Based on 3D Curve Probes , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[14]  Maurice Milgram,et al.  Multi-class Vehicle Type Recognition System , 2008, ANNPR.

[15]  Fei Gao,et al.  Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking , 2017, IEEE Transactions on Cybernetics.

[16]  Hua Huang,et al.  A Hierarchical Scheme for Vehicle Make and Model Recognition From Frontal Images of Vehicles , 2019, IEEE Transactions on Intelligent Transportation Systems.

[17]  Attila Tanács,et al.  Car recognition from frontal images in mobile environment , 2013, 2013 8th International Symposium on Image and Signal Processing and Analysis (ISPA).

[18]  Yu Zhou,et al.  Fine-Grained Vehicle Model Recognition Using A Coarse-to-Fine Convolutional Neural Network Architecture , 2017, IEEE Transactions on Intelligent Transportation Systems.

[19]  Takeo Kato,et al.  Preceding vehicle recognition based on learning from sample images , 2002, IEEE Trans. Intell. Transp. Syst..

[20]  Hyo Jong Lee,et al.  Neural Network Approach to Identify Model of Vehicles , 2006, ISNN.

[21]  I. Zafar,et al.  Localized contourlet features in vehicle make and model recognition , 2009, Electronic Imaging.

[22]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Hyo Jong Lee,et al.  Local Tiled Deep Networks for Recognition of Vehicle Make and Model , 2016, Sensors.

[24]  Timothy F. Cootes,et al.  Analysis of Features for Rigid Structure Vehicle Type Recognition , 2004, BMVC.

[25]  Jianping Fan,et al.  iPrivacy: Image Privacy Protection by Identifying Sensitive Objects via Deep Multi-Task Learning , 2017, IEEE Transactions on Information Forensics and Security.

[26]  T Senthil Kumar,et al.  A modified approach for detecting car in video using feature extraction techniques , 2012 .

[27]  Jun Yu,et al.  Click Prediction for Web Image Reranking Using Multimodal Sparse Coding , 2014, IEEE Transactions on Image Processing.

[28]  Osama Masoud,et al.  Detection and classification of vehicles , 2002, IEEE Trans. Intell. Transp. Syst..

[29]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[30]  Michal Strzelecki,et al.  Hybrid no-propagation learning for multilayer neural networks , 2018, Neurocomputing.

[31]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Michael G. Madden,et al.  Multi-Class and Single-Class Classification Approaches to Vehicle Model Recognition from Images , 2005 .

[33]  Yujian Li,et al.  Vector-kernel convolutional neural networks , 2019, Neurocomputing.

[34]  Jun Yu,et al.  Local Deep-Feature Alignment for Unsupervised Dimension Reduction , 2018, IEEE Transactions on Image Processing.

[35]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[36]  Tatsuya Yoshida,et al.  Vehicle Classification System with Local-Feature Based Algorithn Using CG Model Images , 2002 .

[37]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[38]  Azzedine Boukerche,et al.  Real-Time Vehicle Make and Model Recognition Based on a Bag of SURF Features , 2016, IEEE Transactions on Intelligent Transportation Systems.

[39]  Meng Wang,et al.  Multimodal Deep Autoencoder for Human Pose Recovery , 2015, IEEE Transactions on Image Processing.

[40]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[41]  Nick Pears,et al.  Automatic make and model recognition from frontal images of cars , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[42]  Xiaogang Wang,et al.  Multi-View Perceptron: a Deep Model for Learning Face Identity and View Representations , 2014, NIPS.

[43]  Xiaoou Tang,et al.  A large-scale car dataset for fine-grained categorization and verification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Zhenbing Liu,et al.  An Efficient Method for Vehicle Model Identification via Logo Recognition , 2013, 2013 International Conference on Computational and Information Sciences.

[46]  Qingming Huang,et al.  Spatial Pyramid-Enhanced NetVLAD With Weighted Triplet Loss for Place Recognition , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Eleftherios Kayafas,et al.  Vehicle model recognition from frontal view image measurements , 2011, Comput. Stand. Interfaces.

[48]  Paulo Lobato Correia,et al.  Car recognition based on back lights and rear view features , 2009, 2009 10th Workshop on Image Analysis for Multimedia Interactive Services.