A novel filtering kernel based on difference of derivative Gaussians with applications to dynamic texture representation

Abstract Efficiently representing spatio-temporal features of dynamic textures (DTs) in videos has been restricted due to negative impacts of the well-known issues of environmental changes, illumination, and noise. In order to mitigate those, this paper proposes a new approach for an efficient DT representation by addressing the following novel concepts. Firstly, a novel filtering kernel, called Difference of Derivative Gaussians (DoDG), is introduced for the first time based on high-order derivative of a Gaussian kernel. It allows to point out DoDG-based filtered outcomes which are prominently resistant to noise for DT representation compared to exploiting the conventional Difference of Gaussians (DoG). A new framework in low computational complexity is then presented to take DoDG into account video denoising as an effective preprocessing of DT encoding. Finally, a simple variant of Local Binary Patterns (LBPs) is addressed to extract local features from these DoDG-filtered outcomes for constructing discriminative DoDG-based descriptors in small dimension, expected as one of appreciated solutions for mobile applications. Experimental results for DT recognition have verified that our proposal significantly performs well compared to all non-deep-learning methods, while being very close to deep-learning approaches. Also, ours are eminently better than those based on the traditional DoG.

[1]  F. Bouchara,et al.  Rubik Gaussian-based patterns for dynamic texture classification , 2020, Pattern Recognit. Lett..

[2]  Kuo-Chin Fan,et al.  A Novel Local Pattern Descriptor—Local Vector Pattern in High-Order Derivative Space for Face Recognition , 2014, IEEE Transactions on Image Processing.

[3]  Yong Xu,et al.  Spatiotemporal lacunarity spectrum for dynamic texture classification , 2017, Comput. Vis. Image Underst..

[4]  Yi Fang,et al.  Dynamic texture recognition with video set based collaborative representation , 2016, Image Vis. Comput..

[5]  Payam Saisan,et al.  Dynamic texture recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Nathan Jacobs,et al.  Motion and appearance based background subtraction for freely moving cameras , 2019, Signal Process. Image Commun..

[7]  Vipin Tyagi,et al.  A novel scheme based on local binary pattern for dynamic texture recognition , 2016, Comput. Vis. Image Underst..

[8]  Nuno Vasconcelos,et al.  Classifying Video with Kernel Dynamic Textures , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Thanh Tuan Nguyen,et al.  Prominent Local Representation for Dynamic Textures Based on High-Order Gaussian-Gradients , 2021, IEEE Transactions on Multimedia.

[10]  Richard P. Wildes,et al.  A New Large Scale Dynamic Texture Dataset with Application to ConvNet Understanding , 2018, ECCV.

[11]  Yong Xu,et al.  Dynamic texture classification using dynamic fractal analysis , 2011, 2011 International Conference on Computer Vision.

[12]  Dmitry Chetverikov,et al.  Dynamic Texture Recognition Using Normal Flow and Texture Regularity , 2005, IbPRIA.

[13]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[14]  Oksam Chae,et al.  Spatiotemporal Directional Number Transitional Graph for Dynamic Texture Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  José María Martínez Sanchez,et al.  Stand-alone quality estimation of background subtraction algorithms , 2017, Comput. Vis. Image Underst..

[16]  Hui Ji,et al.  Equiangular Kernel Dictionary Learning with Applications to Dynamic Texture Analysis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Quoc V. Le,et al.  Searching for MobileNetV3 , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Michel Ménard,et al.  Characterization and recognition of dynamic textures based on the 2D+T curvelet transform , 2015, Signal Image Video Process..

[20]  Yong Wang,et al.  Chaotic features for dynamic textures recognition , 2016, Soft Comput..

[21]  Richard P. Wildes,et al.  Spacetime Texture Representation and Recognition Based on a Spatiotemporal Orientation Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Thanh Tuan Nguyen,et al.  Directional dense-trajectory-based patterns for dynamic texture recognition , 2020, IET Comput. Vis..

[23]  Vipin Tyagi,et al.  Dynamic texture recognition based on completed volume local binary pattern , 2016, Multidimens. Syst. Signal Process..

[24]  Zhe Xu,et al.  A Robust Background Initialization Algorithm with Superpixel Motion Detection , 2018, Signal Process. Image Commun..

[25]  Antoni B. Chan,et al.  Clustering dynamic textures with the hierarchical EM algorithm , 2013, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  Baochang Zhang,et al.  Local Derivative Pattern Versus Local Binary Pattern: Face Recognition With High-Order Local Pattern Descriptor , 2010, IEEE Transactions on Image Processing.

[28]  Thanh Phuong Nguyen,et al.  Topological Attribute Patterns for texture recognition , 2016, Pattern Recognit. Lett..

[29]  Thanh Tuan Nguyen,et al.  Directional Beams of Dense Trajectories for Dynamic Texture Recognition , 2018, ACIVS.

[30]  Thanh Tuan Nguyen,et al.  Completed statistical adaptive patterns on three orthogonal planes for recognition of dynamic textures and scenes , 2018, J. Electronic Imaging.

[31]  Thanh Phuong Nguyen,et al.  Spatial Motion Patterns: Action Models from Semi-Dense Trajectories , 2014, Int. J. Pattern Recognit. Artif. Intell..

[32]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[33]  Narendra Ahuja,et al.  Maximum Margin Distance Learning for Dynamic Texture Recognition , 2010, ECCV.

[34]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Minwei Deng Robust human gesture recognition by leveraging multi-scale feature fusion , 2020, Signal Process. Image Commun..

[36]  Qi Tian,et al.  CenterNet: Keypoint Triplets for Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Anil K. Jain,et al.  Unsupervised texture segmentation using Gabor filters , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[38]  Gang Wang,et al.  Optimizing LBP Structure For Visual Recognition Using Binary Quadratic Programming , 2014, IEEE Signal Processing Letters.

[39]  Shervin Rahimzadeh Arashloo Sparse binarised statistical dynamic features for spatio-temporal texture analysis , 2019, Signal Image Video Process..

[40]  Yong Xu,et al.  Scale-space texture description on SIFT-like textons , 2012, Comput. Vis. Image Underst..

[41]  Thanh Tuan Nguyen,et al.  Momental directional patterns for dynamic texture recognition , 2020, Comput. Vis. Image Underst..

[42]  Yu-Long Qiao,et al.  Dynamic Texture Classification Using Multivariate Hidden Markov Model , 2018, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[43]  John C. Woods,et al.  Learn-select-track: An approach to multi-object tracking , 2019, Signal Process. Image Commun..

[44]  Remco C. Veltkamp,et al.  Water detection through spatio-temporal invariant descriptors , 2015, Comput. Vis. Image Underst..

[45]  Xudong Jiang,et al.  Dynamic texture recognition using enhanced LBP features , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[46]  Yaping Lin,et al.  Dynamic Texture Recognition Using Volume Local Binary Count Patterns With an Application to 2D Face Spoofing Detection , 2018, IEEE Transactions on Multimedia.

[47]  Matti Pietikäinen,et al.  Rotation-Invariant Image and Video Description With Local Binary Pattern Features , 2012, IEEE Transactions on Image Processing.

[48]  Loong Fah Cheong,et al.  Synergizing spatial and temporal texture , 2002, IEEE Trans. Image Process..

[49]  Thanh Tuan Nguyen,et al.  Dynamic Texture Representation Based on Hierarchical Local Patterns , 2020, ACIVS.

[50]  Vipin Tyagi,et al.  Improved Weber’s law based local binary pattern for dynamic texture recognition , 2017, Multimedia Tools and Applications.

[51]  Weixin Xie,et al.  Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[52]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[53]  Brian C. Lovell,et al.  Discriminative Non-Linear Stationary Subspace Analysis for Video Classification , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Fugen Zhou,et al.  Stabilization of atmospheric turbulence-distorted video containing moving objects using the monogenic signal , 2018, Signal Process. Image Commun..

[55]  Narciso García,et al.  Human-computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns , 2015, Comput. Vis. Image Underst..

[56]  Vipin Tyagi,et al.  Dynamic texture recognition using multiresolution edge-weighted local structure pattern , 2017, Comput. Electr. Eng..

[57]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Paul F. Whelan,et al.  Convolutional neural network on three orthogonal planes for dynamic texture classification , 2017, Pattern Recognit..

[59]  Thanh Phuong Nguyen,et al.  Local derivative pattern for action recognition in depth images , 2017, Multimedia Tools and Applications.

[60]  Matti Pietikäinen,et al.  Median Robust Extended Local Binary Pattern for Texture Classification , 2016, IEEE Trans. Image Process..

[61]  Thanh Tuan Nguyen,et al.  Volumes of Blurred-Invariant Gaussians for Dynamic Texture Classification , 2019, CAIP.

[62]  Thanh Phuong Nguyen,et al.  Improving texture categorization with biologically-inspired filtering , 2013, Image Vis. Comput..

[63]  Yaping Lin,et al.  Dynamic Texture Classification Using Unsupervised 3D Filter Learning and Local Binary Encoding , 2019, IEEE Transactions on Multimedia.

[64]  Thanh Tuan Nguyen,et al.  Completed local structure patterns on three orthogonal planes for dynamic texture recognition , 2017, 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA).

[65]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[66]  Zhi Zeng,et al.  Adaptive maintenance scheme for codebook-based dynamic background subtraction , 2016, Comput. Vis. Image Underst..

[67]  Yulong Qiao,et al.  Hidden Markov Model Based Dynamic Texture Classification , 2015, IEEE Signal Processing Letters.

[68]  Yong Xu,et al.  Classifying dynamic textures via spatiotemporal fractal analysis , 2015, Pattern Recognit..

[69]  Thanh Tuan Nguyen,et al.  Smooth-Invariant Gaussian Features for Dynamic Texture Recognition , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[70]  Xing Li,et al.  Human-robot interaction based on gesture and movement recognition , 2020, Signal Process. Image Commun..

[71]  Matti Pietikäinen,et al.  Dynamic texture and scene classification by transferring deep image features , 2015, Neurocomputing.

[72]  Nikolaos Grammalidis,et al.  Smoke detection using spatio-temporal analysis, motion modeling and dynamic texture recognition , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[73]  René Vidal,et al.  View-invariant dynamic texture recognition using a bag of dynamical systems , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Wenhao Zhang,et al.  Gender and gaze gesture recognition for human-computer interaction , 2016, Comput. Vis. Image Underst..

[75]  Yong Xu,et al.  Wavelet Domain Multifractal Analysis for Static and Dynamic Texture Classification , 2013, IEEE Transactions on Image Processing.

[76]  Yang Zhao,et al.  Completed Local Binary Count for Rotation Invariant Texture Classification , 2012, IEEE Transactions on Image Processing.

[77]  Dmitry Chetverikov,et al.  Qualitative Characterization of Dynamic Textures for Video Retrieval , 2004, ICCVG.

[78]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[79]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[80]  Yan Huang,et al.  Dynamic Texture Recognition via Orthogonal Tensor Dictionary Learning , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[81]  Hyun Seung Yang,et al.  D3: Recognizing dynamic scenes with deep dual descriptor based on key frames and key segments , 2017, Neurocomputing.

[82]  Shervin Rahimzadeh Arashloo,et al.  Dynamic texture representation using a deep multi-scale convolutional network , 2017, J. Vis. Commun. Image Represent..

[83]  Josef Kittler,et al.  Dynamic Texture Recognition Using Multiscale Binarized Statistical Image Features , 2014, IEEE Transactions on Multimedia.

[84]  Mark J. Huiskes,et al.  DynTex: A comprehensive database of dynamic textures , 2010, Pattern Recognit. Lett..

[85]  Richard P. Wildes,et al.  A Spatiotemporal Oriented Energy Network for Dynamic Texture Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).