QA/QC: challenges and pitfalls facing the microarray community and regulatory agencies

The scientific community has been enthusiastic about DNA microarray technology for pharmacogenomic and toxicogenomic studies in the hope of advancing personalized medicine and drug development. The US Food and Drug Administration has been proactive in promoting the use of pharmacogenomic data in drug development and has issued a draft guidance for the pharmaceutical industry on data submissions. However, many challenges and pitfalls are facing the microarray community and regulatory agencies before microarray data can be reliably applied to support regulatory decision making. Four types of factors (i.e., technical, instrumental, computational and interpretative) affect the outcome of a microarray study, and a major concern about microarray studies has been the lack of reproducibility and accuracy. Intralaboratory data consistency is the foundation of reliable knowledge extraction and meaningful crosslaboratory or crossplatform comparisons; unfortunately, it has not been seriously evaluated and demonstrated in every study. Profound problems in data quality have been observed from analyzing published data sets, and many laboratories have been struggling with technical troubleshooting rather than generating reliable data of scientific significance. The microarray community and regulatory agencies must work together to establish a set of consensus quality assurance and quality control criteria for assessing and ensuring data quality, to identify critical factors affecting data quality, and to optimize and standardize microarray procedures so that biologic interpretation and decision-making are not based on unreliable data. These fundamental issues must be adequately addressed before microarray technology can be transformed from a research tool to clinical practices.

[1]  Weida Tong,et al.  ArrayTrack--supporting toxicogenomic research at the U.S. Food and Drug Administration National Center for Toxicological Research. , 2003, Environmental health perspectives.

[2]  John Quackenbush.  Microarray data normalization and transformation , 2002, Nature Genetics.

[3]  Peter D. Karp,et al.  The MetaCyc Database , 2002, Nucleic Acids Res..

[4]  David Botstein,et al.  BMC Genomics BioMed Central Methodology article Universal Reference RNA as a standard for microarray experiments , 2004 .

[5]  M. R. Fielden,et al.  GP3: GenePix post-processing program for automated analysis of raw microarray data , 2002, Bioinform..

[6]  Huey-Miin Hsueh,et al.  A Generalized Additive Model For Microarray Gene Expression Data Analysis , 2004, Journal of biopharmaceutical statistics.

[7]  T. Speed,et al.  Design issues for cDNA microarray experiments , 2002, Nature Reviews Genetics.

[8]  S. Dudoit,et al.  Normalization for cDNA microarray data: a robust composite method addressing single and multiple slide systematic variation. , 2002, Nucleic acids research.

[9]  B. De Moor,et al.  Comparison and meta-analysis of microarray data: from the bench to the computer desk. , 2003, Trends in genetics : TIG.

[10]  P. S. Pine,et al.  Dye bias correction in dual-labeled cDNA microarray gene expression measurements. , 2004, Environmental health perspectives.

[11]  P. Ghazal,et al.  Experiments using microarray technology: limitations and standard operating procedures. , 2003, The Journal of endocrinology.

[12]  S. Knudsen,et al.  A new non-linear normalization method for reducing variability in DNA microarray experiments , 2002, Genome Biology.

[13]  V. Beneš,et al.  Standardization of protocols in cDNA microarray analysis. , 2003, Trends in biochemical sciences.

[14]  Taesung Park,et al.  Evaluation of normalization methods for microarray data , 2003 .

[15]  L. Lesko,et al.  Microarray data—the US FDA, industry and academia , 2003, Nature Biotechnology.

[16]  R. Tibshirani,et al.  Significance analysis of microarrays applied to the ionizing radiation response , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Sergio Contrino,et al.  ArrayExpress—a public repository for microarray gene expression data at the EBI , 2004, Nucleic Acids Res..

[18]  Geoffrey J McLachlan,et al.  Selection bias in gene extraction on the basis of microarray gene-expression data , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Neil D. Lawrence,et al.  Reducing the variability in cDNA microarray image processing by Bayesian inference , 2004, Bioinform..

[20]  Alex E. Lash,et al.  Gene Expression Omnibus: NCBI gene expression and hybridization array data repository , 2002, Nucleic Acids Res..

[21]  Chen-An Tsai,et al.  Testing for differentially expressed genes with microarray data. , 2003, Nucleic acids research.

[22]  J. Pawley,et al.  Handbook of Biological Confocal Microscopy , 1990, Springer US.

[23]  Shibing Deng,et al.  Cross-site comparison of gene expression data reveals high similarity. , 2004, Environmental health perspectives.

[24]  T. Speed,et al.  Summaries of Affymetrix GeneChip probe level data. , 2003, Nucleic acids research.

[25]  Peter D. Karp,et al.  The Pathway Tools software , 2002, ISMB.

[26]  T. Barrette,et al.  ONCOMINE: a cancer microarray database and integrated data-mining platform. , 2004, Neoplasia.

[27]  Angela Relógio,et al.  Optimization of oligonucleotide-based DNA microarrays. , 2002, Nucleic acids research.

[28]  J. Pronk,et al.  Reproducibility of Oligonucleotide Microarray Transcriptome Analyses , 2002, The Journal of Biological Chemistry.

[29]  Felix Naef,et al.  A study of accuracy, precision in oligonucleotide arrays: extracting more signal at large concentrations , 2002, Bioinform..

[30]  Kristina Hanspers,et al.  Spotted long oligonucleotide arrays for human gene expression analysis. , 2003, Genome research.

[31]  J. John Mann,et al.  Sex genes for genomic analysis in human brain: internal controls for comparison of probe level data extraction. , 2003, BMC Bioinformatics.

[32]  Jason E. Stewart,et al.  Design and implementation of microarray gene expression markup language (MAGE-ML) , 2002, Genome Biology.

[33]  P. Khatri,et al.  Profiling gene expression using onto-express. , 2002, Genomics.

[34]  John Quackenbush,et al.  Universal RNA reference materials for gene expression. , 2004, Clinical chemistry.

[35]  Kris Freeman,et al.  Toxicogenomics Data: The Road to Acceptance , 2004, Environmental health perspectives.

[36]  B. Williams,et al.  Genomic DNA as a cohybridization standard for mammalian microarray measurements. , 2004, Nucleic acids research.

[37]  Chen-An Tsai,et al.  Estimation of False Discovery Rates in Multiple Testing: Application to Gene Microarray Data , 2003, Biometrics.

[38]  Todd R Golub,et al.  Gene expression–based high-throughput screening(GE-HTS) and application to leukemia differentiation , 2004, Nature Genetics.

[39]  P. Tam,et al.  Normalization and analysis of cDNA microarrays using within-array replications applied to neuroblastoma cell response to a cytokine. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[40]  J. D. den Dunnen,et al.  Intensity-based analysis of two-colour microarrays enables efficient and flexible hybridization designs. , 2004, Nucleic acids research.

[41]  Andrew J. Holloway,et al.  Options available—from start to finish—for obtaining data from DNA microarrays II , 2002, Nature Genetics.

[42]  Purvesh Khatri,et al.  Onto-Tools: an ensemble of web-accessible, ontology-based tools for the functional design and interpretation of high-throughput gene expression experiments , 2004, Nucleic Acids Res..

[43]  Richard Simon,et al.  Questions and answers on design of dual-label microarrays for identifying differentially expressed genes. , 2003, Journal of the National Cancer Institute.

[44]  Lucila Ohno-Machado,et al.  Analysis of matched mRNA measurements from two different microarray technologies , 2002, Bioinform..

[45]  J. Weinstein.  'Omic' and hypothesis-driven research in the molecular pharmacology of cancer. , 2002, Current opinion in pharmacology.

[46]  Jiasen Lu,et al.  Assessment of the sensitivity and specificity of oligonucleotide (50mer) microarrays. , 2000, Nucleic acids research.

[47]  E. Hovig,et al.  Profound influence of microarray scanner characteristics on gene expression ratios: analysis and procedure for correction , 2004, BMC Genomics.

[48]  R. Shippy,et al.  An assessment of Motorola CodeLink microarray performance for gene expression profiling applications. , 2002, Nucleic acids research.

[49]  The eMERGE Clinical Annotation Working Group,et al.  Expression profiling — best practices for data generation and interpretation in clinical trials , 2004 .

[50]  L Shi,et al.  Arrays, molecular diagnostics, personalized therapy and informatics , 2001, Expert review of molecular diagnostics.

[51]  G. Churchill.  Fundamentals of experimental design for cDNA microarrays , 2002, Nature Genetics.

[52]  E. Winzeler,et al.  Genomics, gene expression and DNA arrays , 2000, Nature.

[53]  A. Valencia,et al.  A gene network for navigating the literature , 2004, Nature Genetics.

[54]  P. Sorger,et al.  Image metrics in the statistical analysis of DNA microarray data , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[55]  M. Ashburner,et al.  Gene Ontology: tool for the unification of biology , 2000, Nature Genetics.

[56]  Weida Tong,et al.  Technical Issues Involved in Obtaining Reliable Data from Microarray Experiments , 2006 .

[57]  John Aach,et al.  Measuring absolute expression with microarrays with a calibrated reference sample and an extended signal intensity range , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[58]  Richard Simon,et al.  Using DNA microarrays for diagnostic and prognostic prediction , 2003, Expert review of molecular diagnostics.

[59]  T. Jenssen,et al.  A literature network of human genes for high-throughput analysis of gene expression , 2001, Nature Genetics.

[60]  J. Waring,et al.  Interlaboratory evaluation of rat hepatic gene expression changes induced by methapyrilene. , 2004, Environmental health perspectives.

[61]  Ravi Kothapalli,et al.  Microarray results: how accurate are they? , 2002, BMC Bioinformatics.

[62]  S. Sealfon,et al.  Accuracy and calibration of commercial oligonucleotide and custom cDNA microarrays. , 2002, Nucleic acids research.

[63]  L Hunter,et al.  MedMiner: an Internet text-mining tool for biomedical information, with application to gene expression profiling. , 1999, BioTechniques.

[64]  A. Ardekani,et al.  Molecular diagnostics: an FDA perspective , 2003, Expert review of molecular diagnostics.

[65]  Giovanni Parmigiani,et al.  A Cross-Study Comparison of Gene Expression Studies for the Molecular Classification of Lung Cancer , 2004, Clinical Cancer Research.

[66]  Yudong D. He,et al.  Effects of atmospheric ozone on microarray data quality. , 2003, Analytical chemistry.

[67]  Gene Ontology Consortium.  The Gene Ontology (GO) database and informatics resource , 2003 .

[68]  Tsviya Olender,et al.  Human Gene-Centric Databases at the Weizmann Institute of Science: GeneCards, UDB, CroW 21 and HORDE , 2003, Nucleic Acids Res..

[69]  J. Ioannidis,et al.  Predictive ability of DNA microarrays for cancer outcomes and correlates: an empirical assessment , 2003, The Lancet.

[70]  Hugues Bersini,et al.  Integration and cross‐validation of high‐throughput gene expression data: comparing heterogeneous data sets , 2003, FEBS letters.

[71]  James J. Chen,et al.  Multiple‐Testing Strategy for Analyzing cDNA Array Data on Gene Expression , 2004, Biometrics.

[72]  E. Lander,et al.  A molecular signature of metastasis in primary solid tumors , 2003, Nature Genetics.

[73]  R. Lempicki,et al.  Evaluation of gene expression measurements from commercial microarray platforms. , 2003, Nucleic acids research.

[74]  R Kip Guy,et al.  Gene expression as a drug discovery tool , 2004, Nature Genetics.

[75]  M. Radmacher,et al.  Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. , 2003, Journal of the National Cancer Institute.

[76]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[77]  P. Kemmeren,et al.  Monitoring global messenger RNA changes in externally controlled microarray experiments , 2003, EMBO reports.

[78]  Charles Wang,et al.  Multi-class tumor classification by discriminant partial least squares using microarray gene expression data and assessment of classification models , 2004, Comput. Biol. Chem..

[79]  Fuad G. Gwadry,et al.  Comparing cDNA and oligonucleotide array data: concordance of gene expression across platforms for the NCI-60 cancer cells , 2003, Genome Biology.

[80]  Susumu Goto,et al.  The KEGG databases at GenomeNet , 2002, Nucleic Acids Res..

[81]  J. Weinstein,et al.  Transcriptomic analysis of the NCI-60 cancer cell lines. , 2003, Comptes rendus biologies.

[82]  James J. Chen,et al.  Analysis of variance components in gene expression data , 2004, Bioinform..

[83]  Kevin Dobbin,et al.  Statistical Design of Reverse Dye Microarrays , 2003, Bioinform..

[84]  K. Dobbin,et al.  Experimental design of DNA microarray experiments. , 2003, BioTechniques.

[85]  P. Brown,et al.  Large-scale meta-analysis of cancer microarray data identifies common transcriptional profiles of neoplastic transformation and progression. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[86]  T. Barrette,et al.  Meta-analysis of microarrays: interstudy validation of gene expression profiles reveals pathway dysregulation in prostate cancer. , 2002, Cancer research.

[87]  A. Chinnaiyan,et al.  Prostate cancer biomarkers: a current perspective , 2003, Expert review of molecular diagnostics.

[88]  R. Jaques,et al.  Synthetic polypeptides related to corticotrophin acting as histamine liberators. , 1969, Pharmacology.

[89]  Weida Tong,et al.  Decision Forest: Combining the Predictions of Multiple Independent Decision Tree Models , 2003, J. Chem. Inf. Comput. Sci..

[90]  James Lyons-Weiler.  Profound normalisation challenges remain in the analysis of data from microarray experiments. , 2003, Applied bioinformatics.

[91]  Steven C. Lawlor,et al.  GenMAPP, a new tool for viewing and analyzing microarray data on biological pathways , 2002, Nature Genetics.

[92]  Carole L Yauk,et al.  Comprehensive comparison of six microarray technologies. , 2004, Nucleic acids research.

[93]  Kimberly Van Auken,et al.  WormBase: a multi-species resource for nematode biology and genomics , 2004, Nucleic Acids Res..

[94]  May D. Wang,et al.  GoMiner: a resource for biological interpretation of genomic and proteomic data , 2003, Genome Biology.

[95]  P. van Hummelen,et al.  Production of bulk amounts of universal RNA for DNA microarrays. , 2002, BioTechniques.

[96]  Yudong D. He,et al.  Expression profiling using microarrays fabricated by an ink-jet oligonucleotide synthesizer , 2001, Nature Biotechnology.

[97]  Weida Tong,et al.  Development of public toxicogenomics software for microarray data management and analysis. , 2004, Mutation research.

[98]  Duccio Cavalieri,et al.  Standards for Microarray Data , 2002, Science.

[99]  Michael L. Bittner,et al.  Ratio statistics of gene expression levels and applications to microarray data analysis , 2002, Bioinform..

[100]  Diane Gershon,et al.  Microarray technology: An array of opportunities , 2002, Nature.

[101]  Jason E. Stewart,et al.  Minimum information about a microarray experiment (MIAME)—toward standards for microarray data , 2001, Nature Genetics.

[102]  J. Michael Cherry,et al.  Microarray data quality analysis: lessons from the AFGC project , 2004, Plant Molecular Biology.

[103]  Jiang Li,et al.  Differential gene expression patterns revealed by oligonucleotide versus long cDNA arrays. , 2002, Toxicological sciences : an official journal of the Society of Toxicology.

[104]  James J. Chen,et al.  Normalization Methods for Analysis of Microarray Gene-Expression Data , 2003, Journal of biopharmaceutical statistics.

[105]  X. Wang,et al.  Quantitative quality control in microarray image processing and data acquisition. , 2001, Nucleic acids research.

[106]  R. Young,et al.  Biomedical Discovery with DNA Arrays , 2000, Cell.

[107]  Partha S. Vasisht.  Computational Analysis of Microarray Data , 2003 .

[108]  Z. Szallasi,et al.  Sequence-matched probes produce increased cross-platform consistency and more reproducible biological results in microarray-based gene expression measurements. , 2004, Nucleic acids research.

[109]  Karl Kornacker,et al.  Chipping away at the chip bias: RNA degradation in microarray analysis , 2003, Nature Genetics.

[110]  Felix Naef,et al.  Absolute mRNA concentrations from sequence-specific calibration of oligonucleotide arrays. , 2003, Nucleic acids research.

[111]  M. Radmacher,et al.  Design of studies using DNA microarrays , 2002, Genetic epidemiology.