A Novel Sketch Recognition Model based on Convolutional Neural Networks

Deep neural networks have been widely used for visual recognition tasks based on real images as they have proven their efficiency. Unlike real images, sketches exhibit a high level of abstraction as they lack the rich features that the real images contain such as various colors, backgrounds, and environmental detail. Despite all of these shortages and being drawn with just a few strokes, they are still meaningful enough to encompass an appropriate level of meaning. The efficiency of deep neural networks on sketch recognition has been relatively less studied compared to the visual recognition of real images. To experiment with the efficiency of deep neural networks on sketch recognition, a novel sketch recognition model based on Convolutional Neural Networks is proposed in this study. The proposed model consisted of 21 layers and was tuned in an automated manner to find out the best-optimized model. In order to reveal the proposed model’s efficiency in terms of predicting the classes of the given sketches, the model was evaluated on a gold standard sketch dataset, namely, Quick, Draw!. According to the experimental result, the proposed model’s accuracy was calculated as high as 89.53% which outperformed the related work on the same dataset. The key findings that were obtained during the conducted experiments were discussed to shed light on future studies.

[1]  Robert P. W. Duin,et al.  Precision-recall operating characteristic (P-ROC) curves in imprecise environments , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[2]  김종영 구글 TensorFlow 소개 , 2015 .

[3]  Shaogang Gong,et al.  Free-hand sketch recognition by multi-kernel feature learning , 2015, Comput. Vis. Image Underst..

[4]  Babar Shah,et al.  Android malware detection through generative adversarial networks , 2019, Transactions on Emerging Telecommunications Technologies.

[5]  Daniel Riccio,et al.  A Deep Learning Approach for Breast Invasive Ductal Carcinoma Detection and Lymphoma Multi-Classification in Histological Images , 2019, IEEE Access.

[6]  James Hays,et al.  The sketchy database , 2016, ACM Trans. Graph..

[7]  方华 google,我,萨娜 , 2006 .

[8]  Ameya Prabhu,et al.  Distribution-Aware Binarization of Neural Networks for Sketch Recognition , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[9]  Bo Zhao,et al.  Sketch Recognition with Deep Visual-Sequential Fusion Model , 2017, ACM Multimedia.

[10]  Tao Xiang,et al.  Sketch-a-Net that Beats Humans , 2015, BMVC.

[11]  Fang Wang,et al.  Sketch-based 3D shape retrieval using Convolutional Neural Networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Travis E. Oliphant,et al.  Guide to NumPy , 2015 .

[13]  Tao Xiang,et al.  Sketch-a-Net: A Deep Neural Network that Beats Humans , 2017, International Journal of Computer Vision.

[14]  Ivo Düntsch,et al.  Confusion Matrices and Rough Set Data Analysis , 2019, Journal of Physics: Conference Series.

[15]  Nikhil Ketkar,et al.  Deep Learning with Python , 2017 .

[16]  David Tall,et al.  Abstraction as a natural process of mental compression , 2007 .

[17]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[18]  Tinne Tuytelaars,et al.  Sketch classification and classification-driven analysis using Fisher vectors , 2014, ACM Trans. Graph..

[19]  Sherjil Ozair,et al.  SketchTransfer: A Challenging New Task for Exploring Detail-Invariance and the Abstractions Learned by Deep Networks , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[20]  Anima Majumder,et al.  Automatic Facial Expression Recognition System Using Deep Network-Based Data Fusion , 2018, IEEE Transactions on Cybernetics.

[21]  Marian Stewart Bartlett,et al.  Exemplar Hidden Markov Models for classification of facial expressions in videos , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Lei Li,et al.  Sketch-R2CNN: An Attentive Network for Vector Sketch Recognition , 2018, ArXiv.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[25]  Lina Yao,et al.  Deep Neural Network Hyperparameter Optimization with Orthogonal Array Tuning , 2019, ICONIP.

[26]  Wei Wang,et al.  Effective android malware detection with a hybrid model based on deep autoencoder and convolutional neural network , 2018, Journal of Ambient Intelligence and Humanized Computing.

[27]  Jun Guo,et al.  SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Ravi Kiran Sarvadevabhatla,et al.  Enabling My Robot To Play Pictionary: Recurrent Neural Networks For Sketch Recognition , 2016, ACM Multimedia.

[29]  Peng Xu,et al.  Multigraph Transformer for Free-Hand Sketch Recognition , 2019, IEEE Transactions on Neural Networks and Learning Systems.