Multi-Path Deep CNNs for Fine-Grained Car Recognition

Along with the growing demands of intelligent traffic system, how to recognize the category information of a car from surveillance cameras has been an important task. Fine-grained car recognition is facing challenges mainly due to similar appearance of inter-class car images. Moreover, unlike other forms of object recognition, there are a large quantity of car models, that most other objects do not have, which causes fine-grained car recognition to be of vital importance but challenging. Recent research focuses on training a deep convolution neural network (DCNN) on a large dataset. Typically these methods largely rely on the holistic appearance of cars and may fail to train an optimal DCNN to comprehend information from multiple parts of a car. To address this issue, we carefully measure the effectiveness of different parts of a car and highlight the importance of car fronts for fine-grained car recognition. Then, considering multiple important parts from cars, we propose a novel multi-path DCNN model which is equipped with a 3-branch deep convolutional network to better exploit holistic as well as part information for fine-grained car recognition. Multiple parts from cars provide complementary information to boost performance. To further facilitate research on fine-grained car recognition, we also collect a large-scale dataset named “Multiple Parts from Cars” (MPF-Cars) which contains category annotation as well as car part information. We evaluate our proposed multi-path DCNN on MPF-Cars and another benchmark dataset CarFlag-563. Experiments demonstrate the effectiveness of our proposed method.

[1]  Chonghui Guo,et al.  A text semantic topic discovery method based on the conditional co-occurrence degree , 2019, Neurocomputing.

[2]  Jinjia Peng,et al.  Discriminative Feature and Dictionary Learning with Part-aware Model for Vehicle Re-identification , 2020, Neurocomputing.

[3]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[4]  Meena Abdelmaseeh,et al.  Car Make and Model recognition combining global and local cues , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[5]  Tao Mei,et al.  Learning Multi-attention Convolutional Neural Network for Fine-Grained Image Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Tao Mei,et al.  Learning Rich Part Hierarchies With Progressive Attention Networks for Fine-Grained Image Recognition , 2020, IEEE Transactions on Image Processing.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ridha Soua,et al.  Improving Traffic Flow Prediction With Weather Information in Connected Cars: A Deep Learning Approach , 2016, IEEE Transactions on Vehicular Technology.

[9]  Cordelia Schmid,et al.  Segmentation Driven Object Detection with Fisher Vectors , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Richard Szeliski,et al.  Car make and model recognition using 3D curve alignment , 2014, IEEE Winter Conference on Applications of Computer Vision.

[11]  Guosheng Lin,et al.  Deep convolutional neural fields for depth estimation from a single image , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Shaoguo Wen,et al.  Fine-Grained Vehicle Classification With Channel Max Pooling Modified CNNs , 2019, IEEE Transactions on Vehicular Technology.

[14]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[15]  Lin Feng,et al.  Learning a Distance Metric by Balancing KL-Divergence for Imbalanced Datasets , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[16]  Jonathan Krause,et al.  3D Object Representations for Fine-Grained Categorization , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[17]  Nei Kato,et al.  The Deep Learning Vision for Heterogeneous Network Traffic Control: Proposal, Challenges, and Future Perspective , 2017, IEEE Wireless Communications.

[18]  Subhransu Maji,et al.  Bilinear CNN Models for Fine-Grained Visual Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Larry S. Davis,et al.  Jointly Optimizing 3D Model Fitting and Fine-Grained Classification , 2014, ECCV.

[20]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Lenan Wu,et al.  Automatic Modulation Classification: A Deep Learning Enabled Approach , 2018, IEEE Transactions on Vehicular Technology.

[22]  Trevor Darrell,et al.  Pose pooling kernels for sub-category recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Huibing Wang,et al.  Deep CNNs With Spatially Weighted Pooling for Fine-Grained Car Recognition , 2017, IEEE Transactions on Intelligent Transportation Systems.

[24]  Chunhua Shen,et al.  Cross-Convolutional-Layer Pooling for Image Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Nei Kato,et al.  On Removing Routing Protocol from Future Wireless Networks: A Real-time Deep Learning Approach for Intelligent Traffic Control , 2018, IEEE Wireless Communications.

[26]  Xiaoou Tang,et al.  A large-scale car dataset for fine-grained categorization and verification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Nei Kato,et al.  Networking and Communications in Autonomous Driving: A Survey , 2019, IEEE Communications Surveys & Tutorials.

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[30]  Jonathan Krause,et al.  Fine-grained recognition without part annotations , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Larry S. Davis,et al.  Learning a Discriminative Filter Bank Within a CNN for Fine-Grained Recognition , 2016, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[33]  Jie Cao,et al.  Dual Cross-Entropy Loss for Small-Sample Fine-Grained Vehicle Classification , 2019, IEEE Transactions on Vehicular Technology.

[34]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[35]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[36]  Ling Shao,et al.  Efficient Feature Selection and Classification for Vehicle Detection , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Dongyan Chen,et al.  Attribute-Guided Feature Learning Network for Vehicle Reidentification , 2020, IEEE MultiMedia.

[38]  Cordelia Schmid,et al.  Selection of scale-invariant parts for object class recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[39]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Tiejun Huang,et al.  Deep Relative Distance Learning: Tell the Difference between Similar Vehicles , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Richard Szeliski,et al.  Car make and model recognition using 3D curve alignment , 2014, WACV.

[42]  Qi Wang,et al.  Exploiting effects of parts in fine-grained categorization of vehicles , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[43]  T.R. Lim,et al.  Car recognition using Gabor filter feature extraction , 2002, Asia-Pacific Conference on Circuits and Systems.

[44]  Jing Zhang,et al.  Semantic Discriminative Metric Learning for Image Similarity Measurement , 2016, IEEE Transactions on Multimedia.

[45]  Nei Kato,et al.  State-of-the-Art Deep Learning: Evolving Machine Intelligence Toward Tomorrow’s Intelligent Network Traffic Control Systems , 2017, IEEE Communications Surveys & Tutorials.

[46]  Jindong Tan,et al.  Recognition of Car Makes and Models From a Single Traffic-Camera Image , 2015, IEEE Transactions on Intelligent Transportation Systems.