Online streaming feature selection using adapted Neighborhood Rough Set

Abstract Online streaming feature selection, as a new approach which deals with feature streams in an online manner, has attracted much attention in recent years and played a critical role in dealing with high-dimensional problems. However, most of the existing online streaming feature selection methods need the domain information before learning and specifying the parameters in advance. It is hence a challenge to select unified and optimal parameters before learning for all different types of data sets. In this paper, we define a new Neighborhood Rough Set relation with adapted neighbors named the Gap relation and propose a new online streaming feature selection method based on this relation, named OFS-A3M. OFS-A3M does not require any domain knowledge and does not need to specify any parameters in advance. With the “maximal-dependency, maximal-relevance and maximal-significance” evaluation criteria, OFS-A3M can select features with high correlation, high dependency and low redundancy. Experimental studies on fifteen different types of data sets show that OFS-A3M is superior to traditional feature selection methods with the same numbers of features and state-of-the-art online streaming feature selection algorithms in an online manner.

[1]  Xindong Wu,et al.  LOFS: Library of Online Streaming Feature Selection , 2016, Knowl. Based Syst..

[2]  Guoyin Wang,et al.  Online Streaming Feature Selection Based on Conditional Information Entropy , 2017, 2017 IEEE International Conference on Big Knowledge (ICBK).

[3]  James Theiler,et al.  Online Feature Selection using Grafting , 2003, ICML.

[4]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[5]  Pradipta Maji,et al.  Rough set based maximum relevance-maximum significance criterion and Gene selection from microarray data , 2011, Int. J. Approx. Reason..

[6]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[7]  Pablo A. Estévez,et al.  A review of feature selection methods based on mutual information , 2013, Neural Computing and Applications.

[8]  Qinghua Hu,et al.  Neighborhood rough set based heterogeneous feature subset selection , 2008, Inf. Sci..

[9]  Xindong Wu,et al.  Online feature selection for high-dimensional class-imbalanced data , 2017, Knowl. Based Syst..

[10]  Huang Qinghua,et al.  Numerical Attribute Reduction Based on Neighborhood Granulation and Rough Approximation , 2008 .

[11]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[12]  Dominik Slezak,et al.  A framework for learning and embedding multi-sensor forecasting models into a decision support system: A case study of methane concentration in coal mines , 2018, Inf. Sci..

[13]  Rong Jin,et al.  Online Feature Selection and Its Applications , 2014, IEEE Transactions on Knowledge and Data Engineering.

[14]  Luis de Marcos,et al.  Distributed ReliefF-based feature selection in Spark , 2018, Knowledge and Information Systems.

[15]  Dominik Slezak,et al.  Recent Advances in Decision Bireducts: Complexity, Heuristics and Streams , 2013, RSKT.

[16]  Andrzej Skowron,et al.  Rough set methods in feature selection and recognition , 2003, Pattern Recognit. Lett..

[17]  Calton Pu,et al.  Evolutionary study of web spam: Webb Spam Corpus 2011 versus Webb Spam Corpus 2006 , 2012, 8th International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom).

[18]  Qinghua Hu,et al.  Mixed feature selection based on granulation and approximation , 2008, Knowl. Based Syst..

[19]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[20]  Hao Wang,et al.  Online Feature Selection with Streaming Features , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jiawei Han,et al.  Generalized Fisher Score for Feature Selection , 2011, UAI.

[22]  Jing Zhou,et al.  Streamwise Feature Selection , 2006, J. Mach. Learn. Res..

[23]  Chris H. Q. Ding,et al.  Stable feature selection via dense feature groups , 2008, KDD.

[24]  Peng Zhou,et al.  A New Online Feature Selection Method Using Neighborhood Rough Set , 2017, 2017 IEEE International Conference on Big Knowledge (ICBK).

[25]  Mohammad Masoud Javidi,et al.  Streamwise feature selection: a rough set method , 2018, Int. J. Mach. Learn. Cybern..

[26]  Da Ruan,et al.  Neighborhood rough sets for dynamic data mining , 2012, Int. J. Intell. Syst..

[27]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Zhaohui Wu,et al.  A novel ensemble-based wrapper method for feature selection using extreme learning machine and genetic algorithm , 2018, Knowledge and Information Systems.

[29]  Xue-wen Chen,et al.  Combating the Small Sample Class Imbalance Problem Using Feature Selection , 2010, IEEE Transactions on Knowledge and Data Engineering.

[30]  Mohammad Masoud Javidi,et al.  Online streaming feature selection using rough sets , 2016, Int. J. Approx. Reason..

[31]  Jianzhong Li,et al.  A stable gene selection in microarray data analysis , 2006, BMC Bioinformatics.

[32]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[33]  Xindong Wu,et al.  Towards Scalable and Accurate Online Feature Selection for Big Data , 2014, 2014 IEEE International Conference on Data Mining.

[34]  H. Hannah Inbarani,et al.  PSO-based feature selection and neighborhood rough set-based classification for BCI multiclass motor imagery task , 2017, Neural Computing and Applications.

[35]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[36]  Maria Cláudia Reis Cavalcanti,et al.  Automatic feature selection for supervised learning in link prediction applications: a comparative study , 2017, Knowledge and Information Systems.

[37]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[38]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[39]  Z. Pawlak.  Rough Sets: Theoretical Aspects of Reasoning about Data , 1991 .

[40]  Verónica Bolón-Canedo,et al.  On the scalability of feature selection methods on high-dimensional data , 2017, Knowledge and Information Systems.

[41]  Parham Moradi,et al.  OSFSMI: Online stream feature selection method based on mutual information , 2017, Appl. Soft Comput..

[42]  Xindong Wu,et al.  Subkilometer crater discovery with boosting and transfer learning , 2011, TIST.