This communication demonstrates the need for independent validation when an uncertainty calculation procedure is applied to a particular type of measurement problem. A simple measurement scenario is used to highlight differences in the performance of two general methods of uncertainty calculation, one from the Guide to the Expression of Uncertainty in Measurement (GUM) and one from Supplement 1 to the 'Guide to the Expression of Uncertainty in Measurement'—Propagation of Distributions using a Monte Carlo method. The performance of these methods is investigated in terms of the long-run success rate when applied to many simulated measurements in the scenario. An individual application of the method is deemed successful if an uncertainty interval containing the measurand is obtained. The alternative approach to validation taken in the Supplement, that an uncertainty interval calculated by a Monte Carlo method can be used to validate the GUM method, is not consistent with the results of this study.
[1]
B. Hall.
Monte Carlo uncertainty calculations with small-sample estimates of complex quantities
,
2006
.
[2]
Roger Willink,et al.
On using the Monte Carlo method to calculate uncertainty intervals
,
2006
.
[3]
James O. Berger,et al.
The interplay of Bayesian and frequentist analysis
,
2004
.
[4]
Lute Maleki,et al.
Applications of clocks and frequency standards: from the routine to tests of fundamental models
,
2005
.
[5]
R Willink,et al.
A generalization of the Welch–Satterthwaite formula for use with correlated uncertainty components
,
2007
.
[6]
Hari Iyer,et al.
Propagation of uncertainties in measurements using generalized inference
,
2005
.
[7]
S. Standard.
GUIDE TO THE EXPRESSION OF UNCERTAINTY IN MEASUREMENT
,
2006
.
[8]
R Willink.
Principles of probability and statistics for metrology
,
2006
.