A dyadic multi-resolution deep convolutional neural wavelet network for image classification

For almost the past four decades, image classification has gained a lot of attention in the field of pattern recognition due to its application in various fields. Given its importance, several approaches have been proposed up to now. In this paper, we will present a dyadic multi-resolution deep convolutional neural wavelets’ network approach for image classification. This approach consists of performing the classification of one class versus all the other classes of the dataset by the reconstruction of a Deep Convolutional Neural Wavelet Network (DCNWN). This network is based on the Neural Network (NN) architecture, the Fast Wavelet Transform (FWT) and the Adaboost algorithm. It consists, first, of extracting features using the FWT based on the Multi-Resolution Analysis (MRA). These features are used to calculate the inputs of the hidden layer. Second, those inputs are filtered by using the Adaboost algorithm to select the best ones corresponding to each image. Third, we create an AutoEncoder (AE) using wavelet networks of all images. Finally, we apply a pooling for each hidden layer of the wavelet network to obtain a DCNWN that permits the classification of one class and rejects all other classes of the dataset. Classification rates given by our approach show a clear improvement compared to those cited in this article.

[1]  M. Moraud.  Wavelet Networks , 2018, Foundations of Wavelet Networks and Applications.

[2]  Andrew Zisserman,et al.  A Visual Vocabulary for Flower Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[4]  Jane You,et al.  HSAE: A Hessian regularized sparse auto-encoders , 2016, Neurocomputing.

[5]  Ilya Sutskever,et al.  Learning Recurrent Neural Networks with Hessian-Free Optimization , 2011, ICML.

[6]  Wei Huang,et al.  Multi-feature fusion based spatial pyramid deep neural networks image classification , 2015 .

[7]  Shenghuo Zhu,et al.  Deep Learning of Invariant Features via Simulated Fixations in Video , 2012, NIPS.

[8]  Gershon Elber,et al.  Multiresolution Analysis , 2022 .

[9]  Zaher Al Aghbari,et al.  Gabor Wavelet Recognition Approach for Off-Line Handwritten Arabic Using Explicit Segmentation , 2013, IP&C.

[10]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[11]  Chokri Ben Amar,et al.  A New Semantic Approach for CBIR Based on Beta Wavelet Network Modeling Shape Refined by Texture and Color Features , 2014, IDEAL.

[12]  Yann LeCun,et al.  Learning Invariant Feature Hierarchies , 2012, ECCV Workshops.

[13]  Adel M. Alimi,et al.  Impact of Character Models Choice on Arabic Text Recognition Performance , 2010, 2010 12th International Conference on Frontiers in Handwriting Recognition.

[14]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Chokri Ben Amar,et al.  Fast Learning Algorithm of Wavelet Network Based on Fast Wavelet Transform , 2011, Int. J. Pattern Recognit. Artif. Intell..

[16]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Kensuke Yokoi,et al.  APAC: Augmented PAttern Classification with Neural Networks , 2015, ArXiv.

[18]  Mourad Zaied,et al.  Supervised Image Classification Using Deep Convolutional Wavelets Network , 2015, 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI).

[19]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Yang Bingru,et al.  A Novel Word Based Arabic Handwritten Recognition System Using SVM Classifier , 2011, ICEC 2011.

[21]  László Tóth,et al.  Convolutional deep maxout networks for phone recognition , 2014, INTERSPEECH.

[22]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Chokri Ben Amar,et al.  Beta wavelets. Synthesis and application to lossy image compression , 2005, Adv. Eng. Softw..

[24]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[25]  Chokri Ben Amar,et al.  Dyadic Multi-resolution Analysis-Based Deep Learning for Arabic Handwritten Character Classification , 2015, 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI).

[26]  Jing Wang,et al.  A fast deep learning system using GPU , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[27]  Wim Sweldens,et al.  An Overview of Wavelet Based Multiresolution Analyses , 1994, SIAM Rev..

[28]  Ahmed Shaker,et al.  Structure-Based Neural Network Classification for Panchromatic IKONOS Image Using Wavelet-Based Features , 2011, 2011 Eighth International Conference Computer Graphics, Imaging and Visualization.

[29]  John Daugman,et al.  Demodulation by Complex-Valued Wavelets for Stochastic Pattern Recognition , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[30]  Rashad Al-Jawfi,et al.  Handwriting Arabic character recognition LeNet using neural network , 2009, Int. Arab J. Inf. Technol..

[31]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[32]  Mourad Zaied,et al.  A deep convolutional neural wavelet network to supervised Arabic letter image classification , 2015, 2015 15th International Conference on Intelligent Systems Design and Applications (ISDA).

[33]  James Martens,et al.  Deep learning via Hessian-free optimization , 2010, ICML.

[34]  Harold H. Szu,et al.  Neural network adaptive wavelets for signal representation and classification , 1992 .

[35]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[36]  Chokri Ben Amar,et al.  Deep learning with shallow architecture for image classification , 2015, 2015 International Conference on High Performance Computing & Simulation (HPCS).

[37]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[38]  Gernot A. Fink,et al.  Markov models for offline handwriting recognition: a survey , 2009, International Journal on Document Analysis and Recognition (IJDAR).

[39]  Bo Zhao,et al.  Fast low rank representation based spatial pyramid matching for image classification , 2014, Knowl. Based Syst..

[40]  Chokri Ben Amar,et al.  A Novel Approach for Face Recognition Based on Fast Learning Algorithm and Wavelet Network Theory , 2011, Int. J. Wavelets Multiresolution Inf. Process..

[41]  Weiyang Zhou,et al.  Verification of the nonparametric characteristics of backpropagation neural networks for image classification , 1999, IEEE Trans. Geosci. Remote. Sens..

[42]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[43]  Weifeng Liu,et al.  Canonical correlation analysis networks for two-view image recognition , 2017, Inf. Sci..

[44]  Cheng-Yuan Liou,et al.  Autoencoder for words , 2014, Neurocomputing.

[45]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[47]  Chokri Ben Amar,et al.  FBWN: An architecture of fast beta wavelet networks for image classification , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[48]  Thomas Martinetz,et al.  Deep convolutional neural networks as generic feature extractors , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[49]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[50]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[51]  Hossein Mobahi,et al.  Deep Learning via Semi-supervised Embedding , 2012, Neural Networks: Tricks of the Trade.

[52]  Jason Weston,et al.  Deep learning via semi-supervised embedding , 2008, ICML '08.

[53]  Y-Lan Boureau,et al.  Learning Convolutional Feature Hierarchies for Visual Recognition , 2010, NIPS.

[54]  Yagyensh C. Pati,et al.  Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations , 1993, IEEE Trans. Neural Networks.

[55]  Quoc V. Le,et al.  On optimization methods for deep learning , 2011, ICML.

[56]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .