Talk handout at EUPIGCLASS Workshop, 22-23 May 2000 in Lelystad
Poul Thyregod,
Departement of Mathematical Modelling,
Technical University of Denmark,
DK 2800 Lyngby, Denmark
pt@imm.dtu.dk
Basic documents:
Guide to the expression of Uncertainty in Measurement (GUM):
first edition 1993, corrected and reprinted 1995; International Organization
for Standardization (Geneva, Switzerland).
Developed jointly by ISO, IEC, OIML, IFCC, BIPM, IUPAC and IUPAP
ISO 5725, Accuracy (trueness and precision) of measurement methods and results:
International Organization for Standardization (Geneva, Switzerland)
Six parts,
Part 1: General principles and definitions
Part 2: Basic method for the determination of repeatability and reproducibility of a standard measurement method
Part 3: Intermediate measures of the precision of a standard measurement method
Part 4: Basic methods for the determination of the trueness of a standard measurement method
Part 5: Alternative methods for the determination of the precision of a standard measurement method
Part 6: Use in practice of accuracy values
ISO 11095 Linear calibration using reference material
Accuracy of a test method
The closeness of agreement between a test result and the accepted reference value (ISO 3534-1, 3.6)
Uncertainty of a measurement result
A parameter associated with the result of a measurement result that characterises the dispersion of values that could reasonably be attributed to the measurand (GUM H.4 )
Bias
The difference between the expectation of the test results and an accepted reference value (ISO 3534-1, 3.8).
Laboratory bias
The difference between the expectation of the test results from a particular laboratory and an accepted reference value (ISO 3534-1, 3.9).
Bias of the measurement method
The difference between the expectation of test results obtained from all laboratories using that method and an accepted reference value (ISO 3534-1, 3.10).
Laboratory component of bias
The difference between the laboratory bias and the bias of the measurement method.
Precision of a test method
The closeness of agreement between independent test results obtained under stipulated conditions (ISO 3534-1, 3.12)
Repeatability
Precision under repeatability conditions (ISO 3534-1, 3.13)
Repeatability conditions
(ISO 3534-1, 3.14) Conditions where independent test results are obtained with the same method on identical test items.
in the same laboratory
by the same operator
using the same equipment
within short intervals of time
Reproducibility
Precision under reproducibility conditions.
Reproducibility conditions
Conditions where test results are obtained with the same method on identical test items
in different laboratories
with different operators
using different equipment
Sources of variation
operator
equipment
wear of equipment
calibration of equipment
environment
time elapsed between measurements
Simple model for measurement result in interlaboratory test:
result = 'true value' + meth.bias + lab.bias + random noise
Laboratory bias need not be constant over time.
Treated in ISO 5725-2:
Identical samples analyzed at different laboratories using the same method.
Duplicate (or more) determinations on each laboratory to assess repeatability
Sets of samples with different levels of measurand to assess dependence upon value of measurand
Same (standardized) method in order to model laboratory bias as random
For samples that are not certified reference material, laboratory bias expresses deviations from 'accepted reference value'
For non-homogenous material, precision includes variation between samples
ISO 5725-2 provides methods for dealing with outliers.
Philosophy:
Method for evaluating and expressing the uncertainty of a measurement result should be:
Universal: Applicable to all kinds of measurements
Internally consistent: directly derivable from the components that contribute to it (independent of their grouping and decomposition)
the quantity expressing uncertainty should be transferable (uncertainty of one result can be used for measurements where the first is used)
Procedures depend on the mathematical expression of the relationship between result, Y, and input quantities X,W,Z,... through a
function
Y = f (X,W,Z,.....)
The function f should contain every quantity, including all corrections and correction factors that can contribute a significant
component of uncertainty to the result of the measurement.
Determine estimated values x,w,z,... for the input quantities and calculate an estimate y, from the functional relationship, f
Evaluate a standard uncertainty (standard deviation) for each input quantity (and covariances for correlated inputs)
Calculate the combined standard uncertainty from the propagation of error formula.
In principle, a bottom up approach:
Identify all influential quantities
correct for known systematic effects.
Assess all sources of uncertainty.
Specify measurand
Identify uncertainty sources (Fishbone diagram)
Simplify by grouping sources covered by existing data quantify grouped components and remaining components
Calculate combined standard uncertainty
Review and if necessary re-evaluate large components
Method validation studies, intermediate precision measures (the factors varied should correspond to the desired influence factors).
Experimental estimation of individual contributions robustness, ruggedness.
Suppliers' information; calibration certificates
Other calibration experiments, under repeatability or reproducibility conditions (specify factors varied and not varied) (calibration estimates coefficients describing relations between deviations from center point).
Quality control data.
From the draft Eurachem Guide Quantifying Uncertainty in Analytical Measurement, clause 5.2:
``In analytical measurement, it is particularly important to distinguish between measurements intended to produce results which are independent of the method used, and those which are not so intended.
The latter are often referred to as empirical methods. The following examples may clarify this point further.
.....EXAMPLE 2
Determinations of ``extractable fat'' may differ substantially, depending on the extraction conditions specified.
Since ``extractable fat'' is entirely dependent on choice of conditions, the method used is empirical.
It is not meaningful to consider correction for bias intrinsic to the method, since the measurand is defined by the method used. Results are generally reported with reference to the method, uncorrected for any bias intrinsic to the method.
The method is considered empirical.
EXAMPLE 3. In circumstances where variations in the substrate, or matrix have large and unpredictable effects, a systematic procedure is often developed with the sole aim of achieving comparability between laboratories measuring the same material. The method may then be adopted as a local, national or international standard on which trading or other decisions are taken, with no intent to obtain an absolute measure of the true amount of analyte present. Correction for method bias or matrix effects are ignored by convention. Results are normally reported uncorrected for matrix bias. The method is considered to be empirical.
5.3 The distinction between empirical and nonempirical (sometimes called rational) is important because it affects the estimation of uncertainty. In the examples above, because of the conventions employed, uncertainties associated with some quite large effects are not relevant in normal use. Due consideration should accordingly be given to whether the results are expected to be dependent upon, or independent of the method in use and only those effects relevant to the result as reported should be included in the uncertainty estimate. ''
http://www.vtt.fi/ket/eurachem/
Eurachem; possibility of downloading the (final) draft second edition of the guide Quantifying Uncertainty in Analytical Measurement
http://www.european-accreditation.org/
European co-operation for Accreditation;
ISO
http://www.jsa.or.jp/eng/standard/secretary/tc69sc6/
Homepage of ISO/TC69/SC6 "Applications of statistical methods/Measurement Methods and results"
http://www.itl.nist.gov/div898/carroll/sc6wg7.htm
Homepage of ISO/TC69/SC6/WG7. Possibility of downloading Draft 2 of ISO/TC69/SC6/WG7 Statistical Methods of Uncertainty Analysis
M. Boulanger, M. Johnson, C. Perruchet, P. Thyregod:
Evolution of International Statistical Standards via Life Cycle of Products and Services,
International Statistical Review, 67, (1999), pp 151-171
Stephen L.R. Ellison:
ISO uncertainty and collaborative trial data,
Accreditation and Quality Assurance 3, (1998), pp 95-100
Vicki J. Barwich, Stephen L.R. Ellison:
The evaluation of measurement uncertainty from method validation studies, Part 1: Description of a laboratory protocol. Accreditation and Quality Assurance 5, (2000), pp 47-53.
Vicki J. Barwich, Stephen L.R. Ellison, Mark J. Q. Rafferty, Rattanjit S. Gill:
The evaluation of measurement uncertainty from method validation studies, Part 2: The practical application of a laboratory protocol.
Accreditation and Quality Assurance 5, (2000), pp 104-113
Ellison, S. L. R.; Williams, Alex :
Measurement uncertainty and its implications for collaborative study method validation and method performance,
Accreditation and Quality Assurance 3, (1998), pp 6-10
European cooperation for Accreditation of Laboratories,
Expression of the Uncertainty of Measurement in Calibration,
Publication EA-4/02 (1999)