Rotation-Invariant Image and Video Description With Local Binary Pattern Features

In this paper, we propose a novel approach to compute rotation-invariant features from histograms of local noninvariant patterns. We apply this approach to both static and dynamic local binary pattern (LBP) descriptors. For static-texture description, we present LBP histogram Fourier (LBP-HF) features, and for dynamic-texture recognition, we present two rotation-invariant descriptors computed from the LBPs from three orthogonal planes (LBP-TOP) features in the spatiotemporal domain. LBP-HF is a novel rotation-invariant image descriptor computed from discrete Fourier transforms of LBP histograms. The approach can be also generalized to embed any uniform features into this framework, and combining the supplementary information, e.g., sign and magnitude components of the LBP, together can improve the description ability. Moreover, two variants of rotation-invariant descriptors are proposed to the LBP-TOP, which is an effective descriptor for dynamic-texture recognition, as shown by its recent success in different application problems, but it is not rotation invariant. In the experiments, it is shown that the LBP-HF and its extensions outperform noninvariant and earlier versions of the rotation-invariant LBP in the rotation-invariant texture classification. In experiments on two dynamic-texture databases with rotations or view variations, the proposed video features can effectively deal with rotation variations of dynamic textures (DTs). They also are robust with respect to changes in viewpoint, outperforming recent methods proposed for view-invariant recognition of DTs.

[1]  Anil K. Jain,et al.  Texture Analysis , 2018, Handbook of Image Processing and Computer Vision.

[2]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  H. Arof,et al.  Circular neighbourhood and 1-D DFT features for texture classification and segmentation , 1998 .

[4]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1999, TOGS.

[5]  Payam Saisan,et al.  Dynamic texture recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Tieniu Tan,et al.  Brief review of invariant texture analysis methods , 2002, Pattern Recognit..

[7]  Matti Pietikäinen,et al.  Outex - new framework for empirical evaluation of texture analysis algorithms , 2002, Object recognition supported by user interaction for service robots.

[8]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2004, International Journal of Computer Vision.

[10]  Barbara Caputo,et al.  Class-Specific Material Categorisation , 2005, ICCV.

[11]  Weixin Xie,et al.  Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[12]  René Vidal,et al.  Optical flow estimation & segmentation of multiple moving dynamic textures , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Dmitry Chetverikov,et al.  A Brief Survey of Dynamic Texture Description and Recognition , 2005, CORES.

[14]  Andrew W. Fitzgibbon,et al.  Shift-Invariant Dynamic Texture Recognition , 2006, ECCV.

[15]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Dmitry Chetverikov,et al.  Analysis and performance evaluation of optical flow features for dynamic texture recognition , 2007, Signal Process. Image Commun..

[17]  Nuno Vasconcelos,et al.  Classifying Video with Kernel Dynamic Textures , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Matti Pietikäinen,et al.  Improving Rotation Invariance of the Volume Local Binary Pattern Operator , 2007, MVA.

[19]  Matti Pietikäinen,et al.  Unsupervised dynamic texture segmentation using local spatiotemporal descriptors , 2008, 2008 19th International Conference on Pattern Recognition.

[20]  Shu Liao,et al.  Dominant Local Binary Patterns for Texture Classification , 2009, IEEE Transactions on Image Processing.

[21]  Matti Pietikäinen,et al.  Recognition of human actions using texture descriptors , 2011, Machine Vision and Applications.

[22]  Nuno Vasconcelos,et al.  Variational layered dynamic textures , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Yunqian Ma,et al.  Event detection using local binary pattern based dynamic textures , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[24]  John J. Soraghan,et al.  Quantitative Analysis of Facial Paralysis Using Local Binary Patterns in Biomedical Videos , 2009, IEEE Transactions on Biomedical Engineering.

[25]  Matti Pietikäinen,et al.  Rotation Invariant Image Description with Local Binary Pattern Histogram Fourier Features , 2009, SCIA.

[26]  Matti Pietikäinen,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON MULTIMEDIA 1 Lipreading with Local Spatiotemporal Descriptors , 2022 .

[27]  Patrick Bouthemy,et al.  Learning mixed-state Markov models for statistical motion texture tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[28]  René Vidal,et al.  View-invariant dynamic texture recognition using a bag of dynamical systems , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Ling Shao,et al.  Human Action Recognition Using LBP-TOP as Sparse Spatio-Temporal Feature Descriptor , 2009, CAIP.

[30]  Matti Pietikäinen,et al.  Dynamic texture synthesis using a spatial temporal descriptor , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[31]  Zhenhua Guo,et al.  Rotation invariant texture classification using adaptive LBP with directional statistical features , 2010, 2010 IEEE International Conference on Image Processing.

[32]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[33]  Zhenhua Guo,et al.  Rotation invariant texture classification using LBP variance (LBPV) with global matching , 2010, Pattern Recognit..

[34]  Paul F. Whelan,et al.  Evaluation of robustness against rotation of LBP, CCR and ILBP features in granite texture classification , 2011, Machine Vision and Applications.