Feature selection via Boolean independent component analysis

We devise a feature selection method in terms of a follow-out utility of a special classification procedure. In turn, we root the latter on binary features which we extract from the input patterns with a wrapper method. The whole contrivance results in a procedure that is progressive in two respects. As for features, first we compute a very essential representation of them in terms of Boolean independent components in order to reduce their entropy. Then we reverse the representation mapping to discover the subset of the original features supporting a successful classification. As for the classification, we split it into two less hard tasks. With the former we look for a clustering of input patterns that satisfies loose consistency constraints and benefits from the conciseness of binary representation. With the latter we attribute labels to the clusters through the combined use of basically linear separators. We implement out the method through a relatively quick numerical procedure by assembling a set of connectionist and symbolic routines. These we toss on the benchmark of feature selection of DNA microarray data in cancer diagnosis and other ancillary datasets.

[1]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[2]  Alberto J. Pérez Jiménez,et al.  Genetic algorithms for linear feature extraction , 2006, Pattern Recognit. Lett..

[3]  Giorgio Valentini,et al.  Bio-molecular cancer prediction with random subspace ensembles of support vector machines , 2005, Neurocomputing.

[4]  J. Barrett,et al.  G1P3, an interferon inducible gene 6-16, is expressed in gastric cancers and inhibits mitochondrial-mediated apoptosis in gastric cancer cell line TMK-1 cell , 2005, Cancer Immunology, Immunotherapy.

[5]  Loris Nanni,et al.  Evolved Feature Weighting for Random Subspace Classifier , 2008, IEEE Transactions on Neural Networks.

[6]  Thomas A. Darden,et al.  Gene selection for sample classification based on gene expression data: study of sensitivity to choice of parameters of the GA/KNN method , 2001, Bioinform..

[7]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[8]  E. Oja,et al.  Independent Component Analysis , 2001 .

[9]  Gavin C. Cawley,et al.  Gene Selection in Cancer Classification using Sparse Logistic Regression with Bayesian Regularisation , 2006 .

[10]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[11]  Dmitriy Fradkin,et al.  Experiments with random projections for machine learning , 2003, KDD '03.

[12]  M. Ringnér,et al.  Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks , 2001, Nature Medicine.

[13]  C. Perou,et al.  Molecular portraits and the family tree of cancer , 2002, Nature Genetics.

[14]  Marina Vannucci,et al.  Gene selection: a Bayesian variable selection approach , 2003, Bioinform..

[15]  Ron Kohavi,et al.  Wrappers for feature selection , 1997 .

[16]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[17]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .

[18]  David L. Dowe,et al.  MML Inference of Oblique Decision Trees , 2004, Australian Conference on Artificial Intelligence.

[19]  Aidong Zhang,et al.  Feature Extraction from Microarray Expression Data by Integration of Semantic Knowledge , 2007, ICMLA 2007.

[20]  Bruno Apolloni,et al.  Detecting Driving Awareness , 2004, PKDD.

[21]  Nello Cristianini,et al.  Support vector machine classification and validation of cancer tissue samples using microarray expression data , 2000, Bioinform..

[22]  Bruno Apolloni,et al.  BICA: A Boolean Indepenedent Component Analysis Approach , 2008, ICANN.

[23]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[24]  Bruno Apolloni,et al.  From synapses to rules , 2002, Cognitive Systems Research.

[25]  Leon Bobrowski,et al.  Selection of the Linearly Separable Feature Subsets , 2004, ICAISC.

[26]  Jean-Daniel Zucker,et al.  Abstraction, Reformulation and Approximation, 6th International Symposium, SARA 2005, Airth Castle, Scotland, UK, July 26-29, 2005, Proceedings , 2005, SARA.

[27]  C. Emmanouilidis,et al.  A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[28]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[29]  Shutao Li,et al.  Wavelet-Based Feature Extraction for Microarray Data Classification , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[30]  Michael I. Jordan,et al.  Feature selection for high-dimensional genomic microarray data , 2001, ICML.

[31]  C CawleyGavin,et al.  Gene selection in cancer classification using sparse logistic regression with Bayesian regularization , 2006 .

[32]  I. Jolliffe.  Principal Component Analysis , 2002 .

[33]  M. Ringnér,et al.  Analyzing array data using supervised methods. , 2002, Pharmacogenomics.

[34]  Bruno Apolloni,et al.  PAC Meditation on Boolean Formulas , 2002, SARA.

[35]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[36]  K. Kreutz-Delgado,et al.  Application of concave/Schur-concave functions to the learning of overcomplete dictionaries and sparse representations , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[37]  Hyun-Chul Kim,et al.  Constructing support vector machine ensemble , 2003, Pattern Recognit..

[38]  T. Simić,et al.  Glutathione S-transferase T1-1 activity upregulated in transitional cell carcinoma of urinary bladder. , 2005, Urology.

[39]  Xuesong Lu,et al.  Significance of Gene Ranking for Classification of Microarray Samples , 2006, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[40]  Bruno Apolloni,et al.  A general framework for learning rules from data , 2004, IEEE Transactions on Neural Networks.

[41]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[42]  Xiaodong Lin,et al.  Gene expression Gene selection using support vector machines with non-convex penalty , 2005 .

[43]  Giorgio Valentini,et al.  Discovering multi–level structures in bio-molecular data through the Bernstein inequality , 2008, BMC Bioinformatics.

[44]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[45]  Huan Liu,et al.  Redundancy based feature selection for microarray data , 2004, KDD.

[46]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[47]  E. Lander,et al.  A molecular signature of metastasis in primary solid tumors , 2003, Nature Genetics.

[48]  Christoph F. Eick,et al.  Supervised clustering - algorithms and benefits , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[49]  S. Sathiya Keerthi,et al.  Which Is the Best Multiclass SVM Method? An Empirical Study , 2005, Multiple Classifier Systems.

[50]  Peter Bühlmann,et al.  Boosting for Tumor Classification with Gene Expression Data , 2003, Bioinform..

[51]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[52]  G. Dunteman.  Principal Components Analysis , 1989 .

[53]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[54]  Chen-Fu Chien,et al.  Cluster analysis of genome-wide expression data for feature extraction , 2009, Expert Syst. Appl..

[55]  G. Amorino,et al.  Neuroendocrine cells in prostate cancer. , 2004, Critical reviews in eukaryotic gene expression.

[56]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[57]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[58]  Zoran Obradovic,et al.  Feature Selection Filters Based on the Permutation Test , 2004, ECML.

[59]  Giorgio Valentini,et al.  Gene expression data analysis of human lymphoma using support vector machines and output coding ensembles , 2002, Artif. Intell. Medicine.

[60]  Andrew Hunter,et al.  Feature Selection Using Probabilistic Neural Networks , 2000, Neural Computing & Applications.

[61]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[62]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[64]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[65]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[66]  E. Lander,et al.  Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[67]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[68]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[69]  T. H. Bø,et al.  New feature subset selection procedures for classification of expression profiles , 2002, Genome Biology.

[70]  Marcel Dettling,et al.  BagBoosting for tumor classification with gene expression data , 2004, Bioinform..

[71]  Dimitrios Gunopulos,et al.  Non-linear dimensionality reduction techniques for classification and visualization , 2002, KDD.

[72]  S. Dudoit,et al.  Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data , 2002 .

[73]  Stavros J. Perantonis,et al.  Input Feature Extraction for Multilayered Perceptrons Using Supervised Principal Component Analysis , 1999, Neural Processing Letters.

[74]  U. Alon,et al.  Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[75]  Geoffrey J McLachlan,et al.  Selection bias in gene extraction on the basis of microarray gene-expression data , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[76]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[77]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[78]  M. Schapira,et al.  Use of ceruloplasmin levels to monitor response to therapy and predict recurrence of breast cancer , 1983, Breast Cancer Research and Treatment.

[79]  Bruno Apolloni,et al.  BICA: a Boolean independent component analysis algorithm , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[80]  Huan Liu,et al.  Feature Selection for Clustering , 2000, Encyclopedia of Database Systems.

[81]  Carla Angela Mormino.  Istituto Italiano Degli Attuari , 2006 .

[82]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[83]  Kwong-Sak Leung,et al.  A new gradient-based neural network for solving linear and quadratic programming problems , 2001, IEEE Trans. Neural Networks.

[84]  C. Bode,et al.  Decreased expression of cytochrome P450 protein in non-malignant colonic tissue of patients with colonic adenoma , 2005, BMC gastroenterology.

[85]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[86]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[87]  Samuel Kaski,et al.  Discriminative clustering , 2005, Neurocomputing.

[88]  Bruno Apolloni,et al.  Algorithmic Inference in Machine Learning , 2005, IEEE Transactions on Neural Networks.