Online Feature Selection Based on Fuzzy Clustering and Its Applications

Fuzzy c-means (FCM) clustering has been successfully applied in various pattern recognition areas. While FCM is gaining attention, an important issue arising from these studies is the need to determine which attributes of the data should be used. Answering this question is difficult, because there is no labeled training data available in clustering to guide the search. We present a feature selection for FCM. The advantage of our method is that it is intuitively appealing, avoiding combinatorial searches, and allowing us to prune the feature set. Our method is also adaptable and can change through complex scenes in an online environment. We do not have to wait until all data have been generated before learning begins. Finally, to estimate the model parameters, the gradient method is adopted to minimize the fuzzy objective function with the Kullback-Leibler divergence information. Numerical experiments are presented to demonstrate the robustness and accuracy of our method.

[1]  Anil K. Jain,et al.  Simultaneous feature selection and clustering using mixture models , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Sadaaki Miyamoto,et al.  Fuzzy c-means as a regularization and maximum entropy approach , 1997 .

[5]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[6]  Martial Hebert,et al.  A Measure for Objective Evaluation of Image Segmentation Algorithms , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[7]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Sotirios Chatzis,et al.  A Fuzzy Clustering Approach Toward Hidden Markov Random Field Models for Enhanced Spatially Constrained Image Segmentation , 2008, IEEE Transactions on Fuzzy Systems.

[9]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[10]  Jing Zhang,et al.  Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Katsuhiro HONDA,et al.  Fuzzy c-Means Clustering with Regularization by KL Information , 2001 .

[12]  Stan Sclaroff,et al.  A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Yang Wang,et al.  A dynamic conditional random field model for foreground and shadow segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Tamás Szirányi,et al.  Bayesian Foreground and Shadow Detection in Uncertain Frame Rate Surveillance Videos , 2008, IEEE Transactions on Image Processing.

[15]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[16]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[17]  Jing Hua,et al.  Simultaneous Localized Feature Selection and Model Detection for Gaussian Mixtures , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  D. Hand,et al.  Idiot's Bayes—Not So Stupid After All? , 2001 .

[19]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[20]  Plamen P. Angelov,et al.  Creating Evolving User Behavior Profiles Automatically , 2012, IEEE Transactions on Knowledge and Data Engineering.

[21]  Hidetomo Ichihashi,et al.  Fuzzy c-means clustering with regularization by K-L information , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[22]  Carla E. Brodley,et al.  Unsupervised Feature Selection Applied to Content-Based Retrieval of Lung Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Zezhi Chen,et al.  Self-adaptive Gaussian mixture model for urban traffic monitoring system , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[24]  Sotirios Chatzis,et al.  Factor Analysis Latent Subspace Modeling and Robust Fuzzy Clustering Using $t$-Distributions , 2009, IEEE Transactions on Fuzzy Systems.

[25]  Floriberto Ortiz-Rodríguez,et al.  A method for online pattern recognition of abnormal eye movements , 2011, Neural Computing and Applications.

[26]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Dar-Shyang Lee,et al.  Effective Gaussian mixture learning for video background subtraction , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[29]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[30]  Q. M. Jonathan Wu,et al.  Dynamic Fuzzy Clustering and Its Application in Motion Segmentation , 2013, IEEE Transactions on Fuzzy Systems.

[31]  Hui Zhang,et al.  Bounded generalized Gaussian mixture model , 2014, Pattern Recognit..

[32]  Mubarak Shah,et al.  Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[34]  Aristidis Likas,et al.  Bayesian feature and model selection for Gaussian mixture models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.