Microarray data classified by artificial neural networks.

Systems biology has enjoyed explosive growth in both the number of people participating in this area of research and the number of publications on the topic. The field of systems biology encompasses the in silico analysis of high-throughput data as provided by DNA or protein microarrays. Along with the increasing availability of microarray data, attention is focused on methods of analyzing the expression rates. One important type of analysis is the classification task, for example, distinguishing different types of cell functions or tumors. Recently, interest has been awakened toward artificial neural networks (ANN), which have many appealing characteristics such as an exceptional degree of accuracy. Nonlinear relationships or independence from certain assumptions regarding the data distribution are also considered. The current work reviews advantages as well as disadvantages of neural networks in the context of microarray analysis. Comparisons are drawn to alternative methods. Selected solutions are discussed, and finally algorithms for the effective combination of multiple ANNs are presented. The development of approaches to use ANN-processed microarray data applicable to run cell and tissue simulations may be slated for future investigation.

[1]  T. Sejnowski,et al.  Predicting the secondary structure of globular proteins using neural network models. , 1988, Journal of molecular biology.

[2]  M. Giger,et al.  Malignant and benign clustered microcalcifications: automated feature analysis and classification. , 1996, Radiology.

[3]  Tommy W. S. Chow,et al.  A new method in determining initial weights of feedforward neural networks for training enhancement , 1997, Neurocomputing.

[4]  J V Tu,et al.  Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. , 1996, Journal of clinical epidemiology.

[5]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[6]  P. Royston,et al.  Stability of multivariable fractional polynomial models with selection of variables and transformations: a bootstrap investigation , 2003, Statistics in medicine.

[7]  L. Bottaci,et al.  Artificial neural networks applied to outcome prediction for colorectal cancer patients in separate institutions , 1997, The Lancet.

[8]  Syed Mohsin,et al.  Gene expression profiling for the prediction of therapeutic response to docetaxel in patients with breast cancer , 2003, The Lancet.

[9]  P. Royston,et al.  A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials , 2004, Statistics in medicine.

[10]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[11]  Jude W. Shavlik,et al.  Extracting Refined Rules from Knowledge-Based Neural Networks , 1993, Machine Learning.

[12]  T. H. Bø,et al.  New feature subset selection procedures for classification of expression profiles , 2002, Genome Biology.

[13]  Sung-Bae Cho,et al.  Machine Learning in DNA Microarray Analysis for Cancer Classification , 2003, APBC.

[14]  Ralph Neuneier,et al.  The Observer-Observation Dilemma in Neuro-Forecasting , 1997, NIPS.

[15]  Bagrat R. Amirikian,et al.  What size network is good for generalization of a specific task of interest? , 1994, Neural Networks.

[16]  Thorsteinn S. Rögnvaldsson.  On Langevin Updating in Multilayer Perceptrons , 1994, Neural Computation.

[17]  Tom Tollenaere,et al.  SuperSAB: Fast adaptive back propagation with good scaling properties , 1990, Neural Networks.

[18]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[19]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[20]  Benedikt K. Humpert.  Improving back propagation with a new error function , 1994, Neural Networks.

[21]  Sherif Hashem,et al.  Optimal Linear Combinations of Neural Networks , 1997, Neural Networks.

[22]  Ferdinand Hergert,et al.  Improving model selection by nonconvergent methods , 1993, Neural Networks.

[23]  R. Tibshirani,et al.  Improvements on Cross-Validation: The 632+ Bootstrap Method , 1997 .

[24]  Judith A Floyd,et al.  Comparison of Medical Subject Headings and text-word searches in MEDLINE to retrieve studies on sleep in healthy individuals. , 2004, Journal of the Medical Library Association : JMLA.

[25]  S. Dudoit,et al.  Multiple Hypothesis Testing in Microarray Experiments , 2003 .

[26]  Paul W. Munro,et al.  Competition Among Networks Improves Committee Performance , 1996, NIPS.

[27]  Peter L. Bartlett,et al.  Vapnik-Chervonenkis Dimension Bounds for Two- and Three-Layer Networks , 1993, Neural Computation.

[28]  Alan F. Murray,et al.  Synaptic weight noise during multilayer perceptron training: fault tolerance and training improvements , 1993, IEEE Trans. Neural Networks.

[29]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[31]  Siegfried J. Pöppl,et al.  ACMD: A Practical Tool for Automatic Neural Net Based Learning , 2001, ISMDA.

[32]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[33]  Christian A. Rees,et al.  Systematic variation in gene expression patterns in human cancer cell lines , 2000, Nature Genetics.

[34]  W Penny,et al.  Neural Networks in Clinical Medicine , 1996, Medical decision making : an international journal of the Society for Medical Decision Making.

[35]  Christian A. Rees,et al.  Molecular portraits of human breast tumours , 2000, Nature.

[36]  Sung-Bae Cho,et al.  Data Mining For Gene Expression Profiles From Dna Microarray , 2003, Int. J. Softw. Eng. Knowl. Eng..

[37]  J. Stephen Judd,et al.  Optimal stopping and effective machine complexity in learning , 1993, Proceedings of 1995 IEEE International Symposium on Information Theory.

[38]  Esther Levin,et al.  Accelerated Learning in Layered Neural Networks , 1988, Complex Syst..

[39]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[40]  Peter J. Park,et al.  A Nonparametric Scoring Algorithm for Identifying Informative Genes from Microarray Data , 2000, Pacific Symposium on Biocomputing.

[41]  Martin Vingron,et al.  Processing and quality control of DNA array hybridization data , 2000, Bioinform..

[42]  Huan Liu,et al.  NeuroLinear: From neural networks to oblique decision rules , 1997, Neurocomputing.

[43]  G. Stormo,et al.  Role of gene expression microarray analysis in finding complex disease genes , 2002, Genetic epidemiology.

[44]  Gabrielle Prefontaine,et al.  Rapid Identification of Escherichia coli Pathotypes by Virulence Gene Detection with DNA Microarrays , 2003, Journal of Clinical Microbiology.

[45]  Matthew T. Freedman,et al.  Reduction of false positives in lung nodule detection using a two-level neural classification , 1996, IEEE Trans. Medical Imaging.

[46]  Constantin F. Aliferis,et al.  A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis , 2004, Bioinform..

[47]  Christian A. Rees,et al.  Distinctive gene expression patterns in human mammary epithelial cells and breast cancers. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[48]  M. Ringnér,et al.  Microarray-based cancer diagnosis with artificial neural networks. , 2003, BioTechniques.

[49]  Etienne Barnard,et al.  A comparative study of optimization techniques for backpropagation , 1994, Neurocomputing.

[50]  S. Dudoit,et al.  Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data , 2002 .

[51]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[52]  Arjen van Ooyen,et al.  Improving the convergence of the back-propagation algorithm , 1992, Neural Networks.

[53]  D S Ginn,et al.  Quality filtering of the clinical literature by librarians and physicians. , 1993, Bulletin of the Medical Library Association.

[54]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[55]  K. Simpson,et al.  An artificial neural network can select patients at high risk of developing progressive IgA nephropathy more accurately than experienced nephrologists. , 1998, Nephrology, dialysis, transplantation : official publication of the European Dialysis and Transplant Association - European Renal Association.

[56]  Sang-Hoon Oh.  Improving the error backpropagation algorithm with a modified error function , 1997, IEEE Trans. Neural Networks.

[57]  A. El‐Solh,et al.  Predicting active pulmonary tuberculosis using an artificial neural network. , 1999, Chest.

[58]  John Quackenbush,et al.  Computational genetics: Computational analysis of microarray data , 2001, Nature Reviews Genetics.

[59]  F. Harrell,et al.  Artificial neural networks improve the accuracy of cancer survival prediction , 1997, Cancer.

[60]  Ah Chung Tsoi,et al.  Efficacy of modified backpropagation and optimisation methods on a real-world medical problem , 1995, Neural Networks.

[61]  Siegfried J. Pöppl,et al.  The 'subsequent artificial neural network' (SANN) approach might bring more classificatory power to ANN-based DNA microarray analyses , 2004, Bioinform..

[62]  Adrian J. Shepherd,et al.  Second-Order Methods for Neural Networks , 1997 .

[63]  Klaus-Robert Müller,et al.  Asymptotic statistical theory of overtraining and cross-validation , 1997, IEEE Trans. Neural Networks.

[64]  W. Baxt,et al.  Prospective validation of artificial neural network trained to identify acute myocardial infarction , 1996, The Lancet.

[65]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[66]  Raymond J Carroll,et al.  DNA Microarray Experiments: Biological and Technological Aspects , 2002, Biometrics.

[67]  David H. Evans,et al.  Typing and Subtyping Influenza Virus Using DNA Microarrays and Multiplex Reverse Transcriptase PCR , 2001, Journal of Clinical Microbiology.

[68]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[69]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[70]  Lutz Prechelt,et al.  Automatic early stopping using cross validation: quantifying the criteria , 1998, Neural Networks.

[71]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[72]  Mohamed S. Kamel,et al.  Feature Based Decision Fusion , 2001, ICAPR.

[73]  Giorgio Valentini,et al.  Cancer recognition with bagged ensembles of support vector machines , 2004, Neurocomputing.

[74]  Nello Cristianini,et al.  Support vector machine classification and validation of cancer tissue samples using microarray expression data , 2000, Bioinform..

[75]  Kunihiko Ito,et al.  Detection and Identification of Mycobacterium Species Isolates by DNA Microarray , 2003, Journal of Clinical Microbiology.

[76]  Yves Grandvalet,et al.  Noise Injection: Theoretical Prospects , 1997, Neural Computation.

[77]  E. Wolski,et al.  Normalization strategies for cDNA microarrays. , 2000, Nucleic acids research.

[78]  Yudong D. He,et al.  A Gene-Expression Signature as a Predictor of Survival in Breast Cancer , 2002 .

[79]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[80]  Padraig Cunningham,et al.  The NeuralBAG algorithm: optimizing generalization performance in bagged neural networks , 1999, ESANN.

[81]  B Parmanto,et al.  Building Clinical Classifiers Using Incomplete Observations – A Neural Network Ensemble for Hepatoma Detection in Patients with Cirrhosis , 1995, Methods of Information in Medicine.

[82]  Chen-An Tsai,et al.  Gene selection for sample classifications in microarray experiments. , 2004, DNA and cell biology.

[83]  Ash A. Alizadeh,et al.  Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling , 2000, Nature.

[84]  Ignacio Requena,et al.  Are artificial neural networks black boxes? , 1997, IEEE Trans. Neural Networks.

[85]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[86]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[87]  Richard W. Titball,et al.  Genome-Wide DNA Microarray Analysis of Francisella tularensis Strains Demonstrates Extensive Genetic Conservation within the Species but Identifies Regions That Are Unique to the Highly Virulent F. tularensis subsp. tularensis , 2003, Journal of Clinical Microbiology.

[88]  J. Mesirov,et al.  Interpreting patterns of gene expression with self-organizing maps: methods and application to hematopoietic differentiation. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[89]  Nir Friedman,et al.  Tissue classification with gene expression profiles. , 2000 .

[90]  Lucila Ohno-Machado,et al.  Logistic regression and artificial neural network classification models: a methodology review , 2002, J. Biomed. Informatics.

[91]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[92]  Wim D'Haeze,et al.  Leishmania genes involved in parasitic infection , 2002, Genome Biology.

[93]  Yoonkyung Lee,et al.  Classification of Multiple Cancer Types by Multicategory Support Vector Machines Using Gene Expression Data , 2003, Bioinform..

[94]  Jean-Pierre Martens,et al.  On the initialization and optimization of multilayer perceptrons , 1994, IEEE Trans. Neural Networks.

[95]  loannis F. Dimopoulos,et al.  Combining Neural Network Models to Predict Spatial Patterns of Airborne Pollutant Accumulation in Soils around an Industrial Point Emission Source , 2004, Journal of the Air & Waste Management Association.

[96]  R. Tibshirani,et al.  Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[97]  Sayan Mukherjee,et al.  Molecular classification of multiple tumor types , 2001, ISMB.

[98]  Carl G. Looney.  Stabilization and speedup of convergence in training feedforward neural networks , 1996, Neurocomputing.

[99]  Martti Juhola,et al.  Treatment of missing data values in a neural network based decision support system for acute abdominal pain , 1998, Artif. Intell. Medicine.

[100]  Harris Drucker,et al.  Improving Performance in Neural Networks Using a Boosting Algorithm , 1992, NIPS.

[101]  T. Poggio,et al.  Multiclass cancer diagnosis using tumor gene expression signatures , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[102]  Alfred J. Lotka,et al.  The frequency distribution of scientific productivity , 1926 .

[103]  G J Barton,et al.  Application of multiple sequence alignment profiles to improve protein secondary structure prediction , 2000, Proteins.

[104]  Neal S. Holter,et al.  Dynamic modeling of gene expression data. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[105]  Derek Partridge,et al.  Ranking Pattern Recognition Features for Neural Networks , 1999 .

[106]  Noboru Murata,et al.  An Integral Representation of Functions Using Three-layered Networks and Their Approximation Bounds , 1996, Neural Networks.

[107]  Jukka Saarinen,et al.  Initializing Weights of a Multilayer Perceptron Network by Using the Orthogonal Least Squares Algorithm , 1995, Neural Computation.

[108]  Soo-Young Lee,et al.  Training Algorithm with Incomplete Data for Feed-Forward Neural Networks , 1999, Neural Processing Letters.

[109]  D. Chalfin,et al.  Neural Networks: A New Tool for Predictive Models , 1996 .

[110]  S. Dudoit,et al.  Normalization for cDNA microarray data: a robust composite method addressing single and multiple slide systematic variation. , 2002, Nucleic acids research.

[111]  Gilbert Reibnegger,et al.  The determination of three subcutaneous adipose tissue compartments in non-insulin-dependent diabetes mellitus women with artificial neural networks and factor analysis , 1999, Artif. Intell. Medicine.

[112]  David J. Hand,et al.  Academic Obsessions and Classification Realities: Ignoring Practicalities in Supervised Classification , 2004 .

[113]  Yu-Bin Yang,et al.  Lung cancer cell identification based on artificial neural network ensembles , 2002, Artif. Intell. Medicine.

[114]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[115]  Ching Y. Suen,et al.  Optimal combinations of pattern classifiers , 1995, Pattern Recognit. Lett..

[116]  Li Song,et al.  Neural network analysis of lymphoma microarray data: prognosis and diagnosis near-perfect , 2003, BMC Bioinformatics.

[117]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[118]  S. Dudoit,et al.  STATISTICAL METHODS FOR IDENTIFYING DIFFERENTIALLY EXPRESSED GENES IN REPLICATED cDNA MICROARRAY EXPERIMENTS , 2002 .

[119]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[120]  Roberto Battiti,et al.  Democracy in neural nets: Voting schemes for classification , 1994, Neural Networks.

[121]  Marti A. Hearst.  Intelligent Connections: Battling with GA-Joe. , 1998 .

[122]  Kenji Baba,et al.  Explicit representation of knowledge acquired from plant historical data using neural network , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[123]  Tianzi Jiang,et al.  A combinational feature selection and ensemble neural network method for classification of gene expression data , 2004, BMC Bioinformatics.

[124]  Mark Schena,et al.  DNA microarrays : a practical approach , 1999 .