Feature selection algorithm based on PDF/PMF area difference

Abstract Purpose The research devoted to feature selection has been studied extensively in the last years and as a result, important results have been obtained. However, a feature selection technique remains a complex problem because of the necessity for its outcomes to be reliable and accurate within different limitations. Nowadays, there is no feature selection algorithm that outperforms any others. Our research is focused on the development of a feature selection algorithm and its comparison with other well-known feature selection techniques. The proposed algorithm is used for searching for the most reliable features. Despite the fact that we used the proposed algorithm in a specific case in our study (catheter detection task), the algorithm can also be used for general purposes as well. Therefore, it can be easily scaled to find the relevant features of the region of interest in other tasks. Methods The proposed algorithm is based on the difference in probability density functions (PDFs) or probability mass functions (PMFs). We introduced a score value to assess how stable a certain feature is. The score value determines the degree of intersection of areas under the density/mass distribution functions. In our case where a catheter is detected, we described the region of interest by dividing a 20-feature set into several groups: morphometric, statistical, intensity-based, textural and geometric. To evaluate the results, we compared the results of the proposed algorithm with state-of-the-art feature selection techniques. The sets of features selected by different algorithms were used to train and test a linear SVM classifier. As an additional estimator, we used a neural network with the cascade-forward backpropagation (CFB). Results The proposed Area Difference Feature Selection (ADFS) algorithm obtained the following accuracy on the intracardiac catheter dataset: 78.7 ± 3.2 % for a 3-feature subset, 82.3 ± 2.8 % for a 6-feature subset and 86.7 ± 1.5 % for a 12-feature subset. According to the obtained accuracy, ADFS was on the list of the three best algorithms. Additionally, we tested the ADFS algorithm on the breast cancer dataset from the UCI machine learning repository and observed that it outperformed other compared algorithms on the 3-feature and 12-feature subsets. Regarding the 6-feature subset, ADFS is inferior in accuracy to only MRMR and MCFS by approximately 1.8 % and 0.3 %, respectively. According to the processing time assessment, the most time-consuming algorithms were FSCM, MRMR, UDFS, and RFFS. In turn, the fastest algorithms were INFS, CFS, and ADFS with the following processing times: 2 ± 4, 2 ± 5 and 6 ± 16 ms. Conclusion By testing and comparing the different feature selection algorithms, the proposed feature selection algorithm is shown to be accurate and effective for various tasks, including medical imaging and visualization.

[1]  Jose A. Ramirez-Hernandez,et al.  Control of a re-entrant line manufacturing model with a reinforcement learning approach , 2007, ICMLA 2007.

[2]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[3]  R Muthukrishnan,et al.  LASSO: A feature selection technique in predictive modeling for machine learning , 2016, 2016 IEEE International Conference on Advances in Computer Applications (ICACA).

[4]  Shou-De Lin,et al.  Feature Engineering and Classifier Ensemble for KDD Cup 2010 , 2010, KDD 2010.

[5]  S.A.M. Baert,et al.  Three-dimensional guide-wire reconstruction from biplane image sequences for integrated display in 3-D vasculature , 2003, IEEE Transactions on Medical Imaging.

[6]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[7]  Marco Zaffalon,et al.  Robust Feature Selection by Mutual Information Distributions , 2002, UAI.

[8]  Yen-Wei Chen,et al.  Feature Selection Using Recursive Feature Elimination for Handwritten Digit Recognition , 2009, 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[9]  Fei Wang,et al.  Tikhonov or Lasso Regularization: Which Is Better and When , 2013, 2013 IEEE 25th International Conference on Tools with Artificial Intelligence.

[10]  Subhabrata Chakraborti,et al.  Nonparametric Statistical Inference , 2011, International Encyclopedia of Statistical Science.

[11]  Marco Cristani,et al.  Infinite Feature Selection , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Gerhart I. Schuëller,et al.  Computational methods in optimization considering uncertainties – An overview , 2008 .

[13]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[14]  Pablo M. Granitto,et al.  SVM Based Feature Selection: Why Are We Using the Dual? , 2010, IBERAMIA.

[15]  Steven D. Pieper,et al.  Model-based Catheter Segmentation in MRI-images , 2017, ArXiv.

[16]  Calvin R. Maurer,et al.  A Linear Time Algorithm for Computing Exact Euclidean Distance Transforms of Binary Images in Arbitrary Dimensions , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  K. Rhode,et al.  Real-time x-ray fluoroscopy-based catheter detection and tracking for cardiac electrophysiology interventions. , 2013, Medical physics.

[18]  Fernand Meyer,et al.  Topographic distance and watershed lines , 1994, Signal Process..

[19]  Ruibin Ma,et al.  Accurate model‐based segmentation of gynecologic brachytherapy catheter collections in MRI‐images , 2017, Medical Image Anal..

[20]  Heinz Handels,et al.  Efficient patient modeling for visuo-haptic VR simulation using a generic patient atlas , 2016, Comput. Methods Programs Biomed..

[21]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[22]  T. Kailath.  The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[23]  Martin Hilbert,et al.  The World’s Technological Capacity to Store, Communicate, and Compute Information , 2011, Science.

[24]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[25]  Jong Cheol Jeong,et al.  Enhanced recursive feature elimination , 2007, ICMLA 2007.

[26]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[27]  Umberto Castellani,et al.  Infinite Latent Feature Selection: A Probabilistic Latent Graph-Based Ranking Approach , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Petros Drineas,et al.  Feature Selection for Ridge Regression with Provable Guarantees , 2016, Neural Computation.

[29]  Giorgio Roffo,et al.  Ranking to Learn and Learning to Rank: On the Role of Ranking in Pattern Recognition Applications , 2017, ArXiv.

[30]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  P. Bonacich.  Power and Centrality: A Family of Measures , 1987, American Journal of Sociology.

[32]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .