Proficiency tests with uncertainty information: Detection of an unknown random effect

Abstract In proficiency tests (PTs), the possible existence of an unknown random effect (such as instability or inhomogeneity of the measured items, or vagueness in the definition of the measurand) that could affect the reported values is a matter of concern, and may influence the performance evaluations in an unfair manner. If an unknown random effect is the dominant source of uncertainty, it is not appropriate to conduct performance evaluations without correcting for that random effect (increasing the uncertainties, correcting the biases or the both). This study presents a statistical method to detect an unknown random effect before the performance evaluation in a PT with uncertainty information. The method is validated through simulations using various types of virtual but possible data sets. Through the application of this method, the applicability of the PT data to the performance evaluation can be checked.

[1]  Raghu N. Kacker,et al.  Classical and Bayesian interpretation of the Birge test of consistency and its generalized version for correlated results from interlaboratory evaluations , 2008 .

[2]  M. Cox.  The evaluation of key comparison data , 2002 .

[3]  Raghu N. Kacker,et al.  Bayesian alternative to the ISO-GUM's use of the Welch–Satterthwaite formula , 2006 .

[4]  R Willink.  Statistical determination of a comparison reference value using hidden errors , 2002 .

[5]  Vinicius Alves Pessanha,et al.  Proposta para acreditação da divisão de produção de painéis sorológicos de Bio-Manguinhos / Fiocruz na norma ISO / IEC 17043 - Conformity Assessment – General Requirements for Proficiency Testing , 2011 .

[6]  Andrew L. Rukhin,et al.  Estimating heterogeneity variance in meta‐analysis , 2013 .

[7]  Akiko Hirai,et al.  KEY COMPARISON: APMP.L-K1: Calibration of gauge blocks by interferometry: Final Report , 2006 .

[8]  Clemens Elster,et al.  Analysis of key comparisons: estimating laboratories' biases by a fixed effects model using Bayesian model averaging , 2010 .

[9]  Clemens Elster,et al.  Application of Bayesian model averaging using a fixed effects model with linear drift for the analysis of key comparison CCM.P-K12 , 2013 .

[10]  G. Mana,et al.  Model selection in the average of inconsistent data: an analysis of the measured Planck-constant values , 2012, 2007.09428.

[11]  Clemens Elster,et al.  Analysis of key comparison data and laboratory biases , 2008 .

[12]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[13]  Hideyuki Tanaka,et al.  Bayesian statistics for determination of the reference value and degree of equivalence of inconsistent comparison data , 2010 .

[14]  Blaza Toman,et al.  Bayesian Approaches to Calculating a Reference Value in Key Comparison Experiments , 2007, Technometrics.

[15]  Maurice G. Cox,et al.  The evaluation of key comparison data: determining the largest consistent subset , 2007 .

[16]  John Mandel,et al.  Consensus Values and Weighting Factors. , 1982, Journal of research of the National Bureau of Standards.

[17]  Raghu N. Kacker,et al.  Bayesian posterior predictive p-value of statistical consistency in interlaboratory evaluations , 2008 .

[18]  Clemens Elster,et al.  On the adjustment of inconsistent data using the Birge ratio , 2014 .

[19]  Leonard Steinborn,et al.  International Organization for Standardization ISO/IEC 17025 General Requirements for the Competence of Testing and Calibration Laboratories , 2004 .

[20]  Blaza Toman,et al.  Laboratory effects models for interlaboratory comparisons , 2009 .

[21]  Hideyuki Tanaka,et al.  THEORY OF AND COMPUTATION PROGRAM FOR DETERMINATION OF THE REFERENCE VALUE IN KEY COMPARISONS BASED ON BAYESIAN STATISTICS , 2012 .

[22]  Clemens Elster,et al.  Objective Bayesian Inference for a Generalized Marginal Random Effects Model , 2016 .

[23]  Wolfgang Wöger,et al.  Removing model and data non-conformity in measurement evaluation , 2000 .

[24]  Yoshiya Terao,et al.  Final report on the CIPM air speed key comparison (CCM.FF-K3) , 2007 .

[25]  H. Heinzl,et al.  A simulation study comparing properties of heterogeneity measures in meta‐analyses , 2006, Statistics in medicine.

[26]  Hideyuki Tanaka,et al.  Proficiency tests with uncertainty information: Extension of the En number for cases with no reference laboratory , 2016 .