ISO procedures for assessment

of bias and precision

of measurements methods

and uncertainty of measurement results

 

Talk handout at EUPIGCLASS Workshop, 22-23 May 2000 in Lelystad

Poul Thyregod,

Departement of Mathematical Modelling,

Technical University of Denmark,

DK 2800 Lyngby, Denmark

pt@imm.dtu.dk

 

Documents on Measurement methods and measurement results:

 

Basic  documents:

Guide to the expression of Uncertainty in Measurement (GUM):

first edition 1993, corrected and reprinted 1995; International Organization

for Standardization (Geneva, Switzerland).

Developed jointly by ISO, IEC, OIML, IFCC, BIPM, IUPAC and IUPAP

 

ISO 5725, Accuracy (trueness and precision) of measurement methods and results:

International Organization for Standardization (Geneva, Switzerland)

Six parts,

Part 1: General  principles and definitions

Part 2: Basic method for the determination of repeatability and reproducibility of a standard measurement method

Part 3: Intermediate measures of the precision of a standard measurement method

Part 4: Basic methods for the determination of the trueness of a standard measurement method

Part 5: Alternative methods for the determination of the precision of a standard measurement method

Part 6: Use in practice of accuracy values

 ISO 11095 Linear calibration using reference material

 

Definitions:

Accuracy of a test method

The closeness of agreement between a test result and the accepted reference value (ISO 3534-1, 3.6)

 

Uncertainty of a measurement result

A parameter associated with the result of a measurement result that characterises the dispersion of values that could reasonably be attributed to the measurand (GUM H.4 )

 

Bias

The difference between the expectation of  the test results and an accepted reference value (ISO 3534-1, 3.8).

 

Laboratory bias

The difference between the expectation of the test results from a particular laboratory and an accepted reference value  (ISO 3534-1, 3.9).

 

Bias of the measurement method 

The difference between the expectation of  test results obtained from all laboratories using that method and an accepted reference value (ISO 3534-1, 3.10).

 

Laboratory component of bias

The difference between the laboratory bias and the bias of the measurement method.

 

Precision of a test method

The closeness of agreement between  independent test results obtained under stipulated conditions  (ISO 3534-1, 3.12)

 

Repeatability 

Precision under repeatability conditions (ISO 3534-1, 3.13)

 

Repeatability conditions 

(ISO 3534-1, 3.14) Conditions where independent test results are obtained with the same method on identical test items.

 

Reproducibility

Precision under reproducibility conditions.

 

Reproducibility conditions

Conditions  where  test results are obtained with the same method on identical test items

 

Sources of variation

 

Simple model for measurement result in interlaboratory test:

result = 'true value' + meth.bias + lab.bias + random noise

 

Laboratory bias need  not be constant over time.

 

Precision experiment to assess trueness and precision:

Treated in ISO 5725-2:

 

Guide to the expression of uncertainty in measurement

Philosophy:

 

Procedure

Procedures depend on the mathematical expression of the relationship between result, Y, and input quantities X,W,Z,... through a

function

Y = f (X,W,Z,.....)

 

The function f should contain every quantity, including all corrections and correction factors that can contribute a significant

component of uncertainty to the result of the measurement.

 

 Determine estimated values x,w,z,... for the input quantities and calculate an estimate y, from the functional relationship, f

 

Evaluate a standard uncertainty (standard deviation) for each input quantity (and covariances for correlated inputs)

 

Calculate the combined standard uncertainty from the propagation of error formula.

In principle, a bottom up approach:

Identify all influential quantities

correct for known systematic effects.

Assess all sources of uncertainty.

 

Steps in uncertainty estimation process:

  1. Specify measurand

  2. Identify uncertainty sources (Fishbone diagram)

  3. Simplify by grouping sources covered by existing data quantify grouped components and remaining components

  4. Calculate combined standard uncertainty

  5. Review and if necessary re-evaluate large components

 

 

Data sources

 

On the measurand

From the draft Eurachem Guide Quantifying Uncertainty in Analytical Measurement, clause 5.2:

 

``In analytical measurement, it is particularly important to distinguish between measurements intended to produce results which are independent of the method used, and those which are not so intended.

The latter are often referred to as empirical methods.  The following examples may clarify this point further.

.....EXAMPLE 2

Determinations of ``extractable fat'' may differ substantially, depending on the extraction conditions specified.

Since ``extractable fat'' is entirely dependent on choice of conditions, the method used is empirical.

It is not meaningful to consider correction for bias intrinsic to the method, since the measurand is defined by the method used. Results are generally reported with reference to the method, uncorrected for any bias intrinsic to the method.

The method is considered empirical.

 

EXAMPLE 3. In circumstances where variations in the substrate, or matrix have large and unpredictable effects, a systematic procedure is often developed with the sole aim of achieving comparability between laboratories measuring the same material. The method may then be adopted as a local, national or international standard on which trading or other decisions are taken, with no intent to obtain an absolute measure of the true amount of analyte present. Correction for method bias or matrix effects are ignored by convention. Results are normally reported uncorrected for matrix bias. The method is considered to be empirical.

 

5.3 The distinction between empirical and nonempirical (sometimes called rational) is important because it affects the estimation of uncertainty. In the examples above, because of the conventions employed, uncertainties associated with some quite large effects are not relevant in normal use. Due consideration should accordingly be given to whether the results are expected to be dependent upon, or independent of the method in use and only those effects relevant to the result as reported should be included in the uncertainty estimate. ''

 

Useful links:

http://www.vtt.fi/ket/eurachem/ 

Eurachem;  possibility of downloading the (final) draft second edition of the guide Quantifying Uncertainty in Analytical Measurement

http://www.european-accreditation.org/ 

European co-operation for Accreditation;

 

http://www.iso.ch

ISO

 

http://www.jsa.or.jp/eng/standard/secretary/tc69sc6/

Homepage of ISO/TC69/SC6  "Applications of statistical methods/Measurement Methods and  results"

 

http://www.itl.nist.gov/div898/carroll/sc6wg7.htm

Homepage of ISO/TC69/SC6/WG7. Possibility of downloading  Draft 2 of ISO/TC69/SC6/WG7 Statistical Methods of Uncertainty Analysis

 

References:

M. Boulanger, M. Johnson, C. Perruchet, P. Thyregod:

Evolution of International Statistical Standards via Life Cycle of Products and Services,

International Statistical Review, 67, (1999), pp 151-171

 

Stephen L.R. Ellison:

ISO uncertainty and collaborative trial data,

Accreditation and Quality Assurance 3, (1998), pp 95-100

 

Vicki J. Barwich, Stephen L.R. Ellison:

The evaluation of measurement uncertainty from method validation studies, Part 1: Description of a laboratory protocol. Accreditation and Quality Assurance 5, (2000), pp 47-53.

 

Vicki J. Barwich, Stephen L.R. Ellison, Mark J. Q. Rafferty, Rattanjit S. Gill:

The evaluation of measurement uncertainty from method validation studies, Part 2: The practical application of a laboratory protocol.

Accreditation and Quality Assurance 5, (2000), pp 104-113

 

Ellison, S. L. R.; Williams, Alex :

Measurement uncertainty and its implications for collaborative study method validation and method performance,

Accreditation and Quality Assurance 3, (1998), pp 6-10

 

European cooperation for Accreditation of Laboratories,

Expression of the Uncertainty of Measurement in Calibration,

Publication EA-4/02 (1999)