Learning from microarray interlaboratory studies: measures of precision for gene expression

BackgroundThe ability to demonstrate the reproducibility of gene expression microarray results is a critical consideration for the use of microarray technology in clinical applications. While studies have asserted that microarray data can be "highly reproducible" under given conditions, there is little ability to quantitatively compare amongst the various metrics and terminology used to characterize and express measurement performance. Use of standardized conceptual tools can greatly facilitate communication among the user, developer, and regulator stakeholders of the microarray community. While shaped by less highly multiplexed systems, measurement science (metrology) is devoted to establishing a coherent and internationally recognized vocabulary and quantitative practice for the characterization of measurement processes.ResultsThe two independent aspects of the metrological concept of "accuracy" are "trueness" (closeness of a measurement to an accepted reference value) and "precision" (the closeness of measurement results to each other). A carefully designed collaborative study enables estimation of a variety of gene expression measurement precision metrics: repeatability, several flavors of intermediate precision, and reproducibility. The three 2004 Expression Analysis Pilot Proficiency Test collaborative studies, each with 13 to 16 participants, provide triplicate microarray measurements on each of two reference RNA pools. Using and modestly extending the consensus ISO 5725 documentary standard, we evaluate the metrological precision figures of merit for individual microarray signal measurement, building from calculations appropriate to single measurement processes, such as technical replicate expression values for individual probes on a microarray, to the estimation and display of precision functions representing all of the probes in a given platform.ConclusionWith only modest extensions, the established metrological framework can be fruitfully used to characterize the measurement performance of microarray and other highly multiplexed systems. Precision functions, summarizing routine precision metrics estimated from appropriately repeated measurements of one or more reference materials as functions of signal level, are demonstrated and merit further development for characterizing measurement platforms, monitoring changes in measurement system performance, and comparing performance among laboratories or analysts.

[1]  E. Filipe,et al.  Some metrological considerations about replicated measurements on standards , 2006 .

[2]  Hongyue Dai,et al.  Rosetta error model for gene expression analysis , 2006, Bioinform..

[3]  P S Pine,et al.  Use of diagnostic accuracy as a metric for evaluating laboratory proficiency with microarray assays using mixed-tissue RNA reference samples. , 2008, Pharmacogenomics.

[4]  Hanlee P. Ji,et al.  The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. , 2006, Nature biotechnology.

[5]  James C. Fuscoe,et al.  Use of a mixed tissue RNA design for performance assessments on multiple microarray formats , 2005, Nucleic acids research.

[6]  Gustavo Stolovitzky,et al.  Comparison of Amersham and Agilent microarray technologies through quantitative noise analysis. , 2006, Omics : a journal of integrative biology.

[7]  Roger Wood,et al.  Using uncertainty functions to predict and specify the performance of analytical methods , 2006 .

[8]  Lei Zhou,et al.  An expression index for Affymetrix GeneChips based on the generalized logarithm , 2005, Bioinform..

[9]  Paul De Bi.  Essential for metrology in chemistry, but not yet achieved: truly internationally understood concepts and associated terms , 2008 .

[10]  J. Eberwine,et al.  Analysis of gene expression in single live neurons. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Walter Liggett,et al.  Normalization and Technical Variation in Gene Expression Measurements , 2006, Journal of research of the National Institute of Standards and Technology.

[12]  John Quackenbush,et al.  Multiple-laboratory comparison of microarray platforms , 2005, Nature Methods.

[13]  D. Duewer,et al.  NIST micronutrients measurement quality assurance program: characterizing the measurement community's performance over time. , 2000, Analytical chemistry.

[14]  Martin Vingron,et al.  Variance stabilization applied to microarray data calibration and to the quantification of differential expression , 2002, ISMB.

[15]  Paul De Bievre.  Essential for metrology in chemistry, but not yet achieved: truly internationally understood concepts and associated terms , 2008 .

[16]  Maqc Consortium.  The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements , 2006, Nature Biotechnology.

[17]  Bryan Frank,et al.  Independence and reproducibility across microarray platforms , 2005, Nature Methods.

[18]  Anton Petrov,et al.  The Global Error Assessment (GEA) model for the selection of differentially expressed genes in microarray data , 2004, Bioinform..

[19]  Gordon K. Smyth,et al.  Statistical analysis of an RNA titration series evaluates microarray precision and sensitivity on a whole-array basis , 2006, BMC Bioinformatics.

[20]  D. Duewer,et al.  Micronutrients Measurement Quality Assurance Program: helping participants use interlaboratory comparison exercise results to improve their long-term measurement performance. , 1999, Analytical chemistry.

[21]  Marc L Salit,et al.  Microarray Scanner Performance Over a Five-Week Period as Measured With Cy5 and Cy3 Serial Dilution Slides , 2008, Journal of research of the National Institute of Standards and Technology.