Using validation data for ISO measurement uncertainty estimation Part 1. Principles of an approach using cause and effect analysis

Transcription

1 Analyst, June 1998, Vol. 123 ( ) 1387 Using validation data for ISO measurement uncertainty estimation Part 1. Principles of an approach using cause and effect analysis Stephen L. R. Ellison *, and Vicki J. Barwick Laboratory of the Government Chemist, Queens Road, Teddington, Middlesex, UK TW11 0LY A strategy for reconciling the information requirements of formal measurement uncertainty estimation principles with data generated from classical analytical method validation studies is described in detail. The approach involves a detailed analysis of influence factors on the analytical results, employing cause and effect analysis, followed by a formal reconciliation stage. The methodology is shown to be consistent with the principles outlined in the ISO Guide to the Expression of Uncertainty in Measurement (GUM), given representative data. Any relevant data may be used, including those obtained from classical validation studies. The relationship between classical validation studies and ISO GUM uncertainty estimation is discussed briefly; it is concluded that the two methodologies are equivalent, subject to additional allowance for terms held constant during validation experiments. Keywords: ISO measurement uncertainty estimation; validation data; cause and effect analysis Comparability is an essential property of analytical results. 1 For consistent interpretation, an estimate of the degree of comparability is equally necessary; 2 in many fields of measurement, this information is provided in the form of a measurement uncertainty estimate. The approach to the estimation of measurement uncertainty described in the ISO Guide to the Expression of Uncertainty in Measurement 3,4 (GUM) is that recommended by the International Bureau of Weights and Measures (BIPM) 5 and sometimes referred to as Recommendation INC-1 (1980). 3 This general approach is based on two main principles. First, contributions to uncertainty may be estimated either by observation of repeated experiments ( Type A evaluation ) or by other means, for example using data such as published reference material uncertainties or, where necessary, professional judgement ( Type B evaluation ). The GUM elaborates on this principle to show that the overall uncertainty figure then explicitly incorporates uncertainties arising from systematic effects in addition to observed random variability, leading to an intrinsically sound indication of comparability of results. Second, separate contributions, however evaluated, are expressed in the form of variances and, where necessary, combined as such. The implementation of these principles developed in the GUM relies on a quantitative model of the measurement system, typically embodied in a mathematical equation including all relevant input factors ( influence factors ). Overall uncertainty is then estimated via the law of propagation of uncertainty, Note that the ISO Guide formally distinguishes not the origins of effects, but their method of estimation, and that only for discussion purposes. We nonetheless retain the use of the terms random and systematic as we will primarily be concerned with the effects leading to uncertainty contributions. following identification and quantification of uncertainties in individual influence factors. The law referred to uses error propagation principles, usually approximated to first order, though it is recognised that higher order terms may be necessary, 3 together with normal principles of combination of variance components and leading to the general expression uyx [(, x x )] ux ( ) sxik (, ) i j n 2 y y y  x i i  xi xk i= 1, n ik, = 1, n iπk K = [ ] + [ ] where the result y(x i,j...)is a function of n parameters x i, x j..., u(y...)its standard uncertainty, u(x i ) the standard uncertainty associated with x i, y/ x i is the partial differential of y with respect to x i (sometimes referred to as the sensitivity coefficient of y with respect to x i ) and s(x,ik) is the covariance between x i and x k. The correlation term does not apply where effects are independent, that is, s(x,ik) = 0. It is accordingly essential to identify and study all the relevant individual effects on the result. This approach will be referred to in this paper as the uncertainty propagation approach (sometimes referred to as a bottom-up approach 6 ). Some successful applications of this methodology in analytical chemistry have been published. 7,8 There are advantages in applying the GUM approach in chemistry, including consistency with other fields of measurement, applicability in the absence of collaborative study data, specific relevance to the individual laboratory and a detailed understanding of contributions to uncertainty which can assist method development. The GUM principles appear to differ substantially from the methodology currently used in analytical chemistry for estimating uncertainty. 6,9 Current practice in establishing confidence and comparability relies on the determination of overall method performance parameters, such as linearity, detection limit, extraction recovery, spike recovery and repeatability, reproducibility and other precision measures. These are obtained during method development and interlaboratory study or by inhouse validation protocols. There is no formal requirement for a full mathematical model. In extreme cases, the GUM uncertainty propagation approach and validation-based approaches can be described as bottom-up and top-down approaches, respectively, 6 although perhaps deconstructive and holistic might be more descriptive terms. Despite this, there is commonality between the formal processes involved, 13 and the ISO GUM states explicitly (Section of ref. 3) that If all the quantities on which the result of a measurement depends are varied, its uncertainty can be evaluated by statistical means, strongly suggesting that a reconciliation between the two is possible in principle. There are, however, significant difficulties in applying the GUM methodology generally in analytical chemistry. 9 In particular, it is common to find that the largest contributions to uncertainty arise from the least predictable effects, such as matrix effects on extraction or (1)

2 1388 Analyst, June 1998, Vol. 123 response, sampling operations, and interferences. In practice, the variability of routine samples prevents complete elucidation of these effects, which accordingly appear as discrete effects per sample and do not lead naturally to continuous differentiable functions. Uncertainties associated with such effects can therefore only be determined by experiment. Such experiments are practical and form the basis for most in-house method validation. However, because they usually study the overall performance of methods, the variation observed includes contributions from some, but not all, other sources of variation, risking double counting when other contributions are studied separately. The same is true of experiments to determine overall bias; all current contributions to bias are included for a particular study, but it is rarely clear which of these might vary in subsequent applications of the same method and thus contribute to uncertainty. Finally, it is common to find that similar information, particularly on precision, is available from several sources, including QC and QA data, proficiency scheme results and more than one performance parameter measurement. The result, when using such data to inform estimates of uncertainty in compliance with the INC-1 recommendation, is substantial difficulty in reconciling the available data with the information required. In a field dominated by an experimental approach, it is important to show how the experiments done may form the basis for an uncertainty estimate that conforms to the definition and fundamental principles of measurement uncertainty as defined by ISO (and in the GUM INC-1 recommendations in particular), and this is the purpose of this paper. We describe and illustrate a structured methodology applied in our laboratory to overcome the difficulties. It will be argued that application of the approach can lead to a full reconciliation of validation studies with the GUM approach, and the advantages and disadvantages of the methodology will be considered. Discussion Principles of approach The strategy has two stages; identifying and structuring the effects on a result, followed by an explicit reconciliation stage to assess the degree to which information available meets the requirement and thus identifying factors requiring further study. The approach is intended to generate an estimate of overall uncertainty, not a detailed quantification of all components. In practice, we effect the necessary structured analysis using a cause and effect diagram (sometimes known as an Ishikawa or fishbone diagram). 14 We have found this presentation to be readily understood and implemented by analysts, and likely to lead to consistent identification of major effects, although clearly any comparable conceptual basis might apply provided that it results in a complete structured list of factors affecting the particular analytical result. We therefore describe the general principles of constructing such a list, illustrated using causeand-effect models. We then describe the use of the resulting list of input factors in reconciling available data with the uncertainty information required. Cause and effect analysis The principles of constructing a cause and effect diagram are described fully elsewhere. 14 A partial cause and effect diagram is shown in Fig. 1. The diagram consists of a hierarchical structure culminating in a single outcome. For our purpose, this outcome is a particular analytical result ( Cholesterol in Fig.1). The branches leading to the outcome are the contributory effects, which include both the results of particular intermediate measurements and values and other factors, such as environmental or matrix effects. Each branch may in turn have further contributory effects. These effects comprise all factors affecting the result, whether variable or constant; uncertainties in any of these effects will clearly contribute to uncertainty in the result. The procedure employed in our laboratory is as follows: 1. Write the complete equation for the result. The parameters in the equation form the main branches of the diagram. (We have found that it is almost always necessary to add a main branch representing nominal correction for overall bias, usually as recovery, and accordingly do so at this stage.) 2. Consider each step of the method and add any further factors to the diagram, working outwards from the main effects. Examples include environmental and matrix effects. 3. For each branch, add contributory factors until effects become sufficiently remote, that is, until effects on the result are negligible. 4. Resolve duplications and rearrange to clarify contributions and group related causes. We have found it convenient to group precision terms at this stage on a separate precision branch. Duplications arise naturally in detailing contributions separately for every input parameter. For example, a run-to-run variability element is always present, and it is not uncommon to find the same instrument used to weigh materials, leading to over-counting of its calibration uncertainties. The following rules are applied to resolve duplication: Cancelling effects: remove both. For example, in a mass by difference, two masses are determined, both subject to the Fig. 1 Partial cause and effect diagram.

3 Analyst, June 1998, Vol balance zero bias. The zero bias will cancel out of the mass by difference, and can be removed from the branches corresponding to the separate weighings. Similar effect, same time: combine into a single input. For example, run-to-run variation on many inputs can be combined into an overall run-to-run precision branch. Some caution is required; specifically, variability in operations carried out individually for every determination can be combined, whereas variability in operations carried out on complete batches (such as instrument calibration) will only be observable in betweenbatch measures of precision. Different instances: re-label. It is common to find similarly named effects which actually refer to different instances of similar measurements. For example, calibration might appear as an effect on more than one intermediate measurement. These must be clearly distinguished before proceeding; for example, two instances of calibration might be relabelled as calibration at m 1, and calibration at m 2. An important example of the above merits further consideration. Overall recovery is studied by applying the method to previously characterised materials. All the parameters measured in normal operation of the method are measured during the recovery experiment. Where a recovery effect appears in a cause and effect analysis, usually as a nominal correction, the primary effects on that nominal correction naturally appear to be the parameters already present as primary effects on the result. The main branches in the diagram thus appear to be duplicated in the recovery estimate branch. However, these are secondary instances representing the magnitudes of the effects operating during the recovery experiment, and should accordingly be distinguished as such. The case of recovery is considered further below. Although this form of analysis does provide an effective tool for structuring the information, it does not lead to uniquely structured lists. For example, in densitometry, temperature may be seen as either a direct effect on the density to be measured, or as an effect on the measured mass of material contained in a density bottle. In practice this does not affect the utility of the method. Provided that all significant effects appear somewhere in the list, the overall methodology remains effective. In terms of method validation, this structured list corresponds to the initial identification of factors for optimisation, and in the GUM approach, would form the basis for the formal uncertainty budget. Once the cause-and-effect analysis is complete, it may be appropriate to return to the original equation for the result and add any new terms (such as temperature) to the equation. However, the reconciliation which follows will often show that additional terms are adequately accounted for; we therefore find it preferable to first conduct the next stage of the analysis. Reconciliation Following elucidation of the effects and parameters influencing the results, a review is conducted to determine qualitatively whether a given factor is duly accounted for by either existing data or experiments planned (we have termed the process reconciliation, by analogy with the reconciliation process applied in financial accounting, which ensures that two accounts represent the same balance). The fundamental assumption underlying this review is that an effect varied representatively during the course of a series of observations needs no further study. In this context, representatively means that the influence parameter has demonstrably taken a distribution of values appropriate to the uncertainty in the parameter in question. For continuous parameters, this may be a permitted range or stated uncertainty; for factors such as sample matrix, this range corresponds to the variety of types permitted or encountered in normal use of the method. This presumption can be justified in terms of GUM principles as follows. Eqn. (1) calculates a standard uncertainty u(y) from contributions u(y i ) = u(x i ) y/ x i. Each value of u(x i ) characterises a dispersion associated with the value x i. The sensitivity coefficient y/ x i may be determined by differentiation (analytically or numerically), or by experiment. Such an experiment normally involves controlled variation of the influence factor and observation of the effect on the result. 3 (Note the resemblance to robustness testing.) Consider an increment Dx i in x i. This will clearly lead to a change Dy in the result given by Dy = y[(x i + Dx i ),x j,... ] 2 y(x j,x, j,...) (2) Sensitivity analysis would normally approximate y/ x i using a calculation such as y/ x i {y[(x i + D x i ), x j,...] 2 y[(x i 2 Dx i ), x j,...]}/2dx i (3) and the uncertainty contribution u(y i ) is calculated from u(y i ) = u(x i ) y/ x i. There are two important corollaries. First, if Dx i = u(x i ), y[(x i + Dx i ),x j,... ] 2 y(x i,x j... ) u(y i ). This result was used to effect by Kragten 15 in a simple numerical method for combining uncertainty contributions. Second, given a distribution f(dx i ) of values of Dx i with dispersion characterised by standard uncertainty u(x i ), the corresponding distribution g(dy i ) of Dy i will be characterised by u(y i ). This is precisely the result required by eqn. (1). Further, the same is true of any other influence parameter. It follows that if any collection of influence parameters x i, x j... is varied representatively, the resulting dispersion of results will include all the relevant contributions y i, y j..., duly combined. Here, it is not necessary to require independence; representative variation in parameters x i, x j will incorporate any correlation and the distribution g(dy i ) will accordingly reflect the correlation terms. In order to demonstrate that a particular contribution to overall uncertainty is adequately incorporated into an observed dispersion of results, it follows that it is sufficient to demonstrate that the distribution of values taken by the influence parameter in the particular experiment is representative of f(dx i ). Some examples will illustrate the principle. For a parameter controlled by specification, such as a temperature range during leaching, the parameter should have varied across the permitted range during a precision study. For a factor such as matrix variation, an appropriate range of different levels (types of matrix) must have been included during a precision experiment capable of detecting matrix effects (for example, an experiment to determine precision associated with recovery). Where a contribution arises from calibration uncertainty or specification, a precision study must incorporate a representative set of different calibration states. A good example is the case of volumetric glassware; if a single item is used throughout an experiment, the relevant contribution will clearly not appear in the observed variation, whereas if several randomly chosen items are used, the effect may reasonably be considered as included. We note, however, that for most cases of analytical uncertainty, calibrations for physical measurements represent negligible contributions. Returning to the reconciliation process, following the arguments above, it becomes a relatively straightforward matter to decide whether a given parameter is sufficiently covered by a given set of data or planned experiment. Where a parameter is already so accounted for, the fact is noted. Finally, the parameters which are not accounted for become the subject of further study, either through planned experimentation, by locating appropriate standing data, or through modelling. The propagation of uncertainty approach described by the GUM is especially valuable here, in allowing estimation of effects not amenable to direct observation by input parameter variation; many systematic effects fall into this category. Note also that the

4 1390 Analyst, June 1998, Vol. 123 reconciliation approach described may, depending on the available data and practicality of large-scale experimentation, lead equally to estimates based on either interlaboratory studies ( top-down ) or GUM uncertainty propagation calculations ( bottom-up ). In practice, we have so far found that most estimates fall between these extremes, in combining overall performance data with additional elements. An example of a reconciled cause and effect study is shown in Fig. 1, which shows a partial diagram (excluding long-term precision contributions) for an internally standardised GC determination of cholesterol in oils and fats. The result, cholesterol concentration c ch in mg per 100 g of material, is given by Ac Rf mis cch = 100 ABm (4) where A c, is the peak area of the cholesterol, A B is the peak area of the betulin internal standard, R f is the response factor of cholesterol with respect to betulin (usually assumed to be 1.00), m IS is the mass of the betulin internal standard (mg) and m is the mass of the sample (g). In addition, a nominal correction for recovery is included in the diagram; to aid treatment of uncertainties arising from this parameter, we extend eqn.(4) to Ac Rf mis 1 cch = 100 ABm R (5) where R is the mean recovery estimate for a sample (which may be 1.0, although there is invariably an associated uncertainty). The diagram shows each parameter in eqn. (5) as a major input to the final result, although the terms A c and A B are shown as contributing to the determination of the ratio A c /A B. Subsequent branches then represent effects operating on each of the separate parameters. [This structure facilitates any subsequent application of eqn. (1) applied to the model in eqn. (5).] Now, if a recovery study including a representative range of matrices and levels of analyte is conducted, and it includes several separate preparations of standards, the dispersion of the recovery results will incorporate all the contributions marked with a tick (]). Note that some are potentially systematic for a particular sample, but (for example, in the case of balance linearity contributions) would vary naturally across a range of different determinations on different samples. Those marked with a cross (3) represent terms unlikely to vary sufficiently, or at all, during a single study. The overall uncertainty can therefore be calculated from the dispersion of recoveries found in the experiment combined with contributions determined for the remaining terms. In this instance, most are calibration terms, and would be quantified using eqn. (1) with reference to uncertainty information contained in standing QA data. It may also be that the calculation is unnecessary where the relevant effects are comparatively small. In other instances, fewer contributions would be accounted for, and some additional terms would need separate study. For example, where only a single material is studied, an additional study of recovery variability with matrix might be necessary (an alternative would be to establish uncertainties for each matrix type). More detailed examples will be published elsewhere. 16 It is important to note that during this process, direct calculation of particular contributions has been avoided almost entirely, particularly for major effects, a combination of which is covered by the experiment postulated. The approach is thus appropriate for estimating overall uncertainty, but not necessarily for obtaining detailed estimates of individual contributions, although the cause and effect analysis can clearly help plan such studies. Further, the risk of double counting is very small; the effects visible in the diagram are unique and do not overlap. Finally, there is no restriction on the type or origin of information which can be reviewed for relevance in this way, provided only that sufficient is known to establish the ranges of influence parameters involved in obtaining the data. Interlaboratory study and the GUM approach Although a full discussion is beyond the scope of this paper, the argument presented above has important implications for the applicability of data from interlaboratory or other whole method performance ( top-down ) studies. 6 Given knowledge of the parameters varied during such a study, it clearly becomes possible, using the reconciliation methodology above, to apply the observed dispersion from an interlaboratory study in a GUM-compliant assessment of overall uncertainty, by combining this figure with additional contributions as necessary to account for factors whose influence does not appear in the observed dispersion. We report such applications elsewhere. 16 Such an estimate naturally combines top-down and bottomup estimates; for this reason, we prefer to restrict these terms to extreme cases. It follows also that if a good estimate of uncertainty is to be obtained directly from a holistic approach, the study should ideally be conducted so as to ensure representative variation of significant influence parameters. In analytical method validation, however, factors such as the analyst, batch of samples and laboratory are considered as significant factors. Such terms do not generally appear in the rigorous mathematical description of a measurement process couched in terms of traceable measurements of input parameters. This apparent dichotomy may be resolved by consideration of the analyst, laboratory, etc., as proxy factors, each representing an individual set of values of the formal influence factors expected by the GUM approach. This is shown schematically in Fig. 2, which represents the distributions of particular influence parameters, x i, x j and x k, implemented in practice by four different analysts, 1 4. Clearly, a sufficiently large population of these analysts, each with a randomly chosen set of influence parameter distributions, would result in a complete, or at least fully representative, distribution of values for each influence parameter. This is the fundamental assumption behind reproducibility estimation by interlaboratory study, although note that reproducibility, by definition, 12 excludes time of analysis or matrix variation as influence factors. It accordingly seems reasonable to consider reproducibility to be a direct estimate of overall measurement uncertainty, subject to consideration of additional factors such as matrix and sample pre-treatment held constant or restricted, deliberately or otherwise, during the study. 4 More restricted studies would normally provide a more restricted distribution for some parameters (as shown for x j in Fig. 2); for example, an in-house study might use a restricted range of temperatures or, more subtly, result in commonality of interpretation, leading to lack of representative coverage in some effects. Fig. 2 Two views of influence factors.

5 Analyst, June 1998, Vol Reliability of estimates Although the general approach described here does not lead necessarily to either whole-method or GUM uncertainty propagation estimates of uncertainty, the nature of current analytical method performance evaluation will tend to dictate estimations based on whole method study, albeit enhanced with specific input factor studies where required. It is accordingly pertinent to consider the relative reliability of the two methodologies. The reliability of uncertainty estimates is hard to quantify without extended study over many years. 17 Nonetheless, some indication of relative reliability can be gained by study of the relative number of degrees of freedom of particular estimates; it may be expected, all other things being equal, that a large number of effective degrees of freedom will lead to more reliable estimates. In a simple, idealised topdown validation study involving a genuinely random selection of all significant inputs, the number of degrees of freedom n val is directly related to the number n of independent experiments performed by n val = n 2 1. In a typical in-house study, n is rarely very large for truly independent experiments, typically of the order of 10. Collaborative trials, although costly, may extend the absolute number of observations substantially, but if interlaboratory effects are significant (as usual), then n is best represented by the number of laboratories participating, typically Typical validation studies, then, will produce estimates with under 20 degrees of freedom. For uncertainty propagation studies, the GUM methodology suggests that effective degrees of freedom n eff are calculated 3 according to 4 4 uc ( y) ui ( y) = (6) n eff i i= ν 1, m with u c (y) being the combined standard uncertainty, u i (y) the contribution from the ith input parameter and n i the degrees of freedom associated with u i (y). For a large number m of comparable, independent contributions u i (y), with similar n i, n eff is close to S ni. For substantially different u i (y), n eff rapidly converges on n max, the number of degrees of freedom for the largest contribution u max (y). Since typical recommendations for routine uncertainty estimation in chemistry (e.g., ref. 4) consider approximately observations for each contribution adequate, it would be expected that where several comparable contributions exist, an uncertainty propagation study should give more degrees of freedom and, by implication, be more reliable than a whole-method study. The number of degrees of freedom in either case reflects the number of observations involved but, for an uncertainty propagation study, the observations are typically of more restricted scope. The relative costs will, of course, depend on the particular experiments required. Given one or two dominant contributions, however (a common eventuality 4 ), the methodologies would tend to give similar degrees of freedom. A clear exception arises where dominant contributions in the uncertainty propagation methodology have large n, which would typically be the case where calibration or reference material uncertainties were dominant. From this viewpoint, therefore, the comparison tends to favour uncertainty propagation approaches where significant contributions are already very well established by prior study, but with few, large contributions which need experimental study, there is no clear reason that either methodology should be more reliable. In essence, the more effort put into obtaining the information, the better the estimate will be in either case. From a non-statistical point of view, the comparison is still less clear. Uncertainty propagation studies require both detailed understanding and substantial effort, which will generally be unavailable in routine analytical testing (for example, because of unpredictability of matrix effects). There are clear risks of under-estimation through ignorance of, and consequent failure to investigate, significant single effects. In general, method performance studies across several analysts and matrices are consequently more likely to give a reliable early indication of real dispersion (including accuracy, if reference materials are available) pending identification and quantitation of the individual effects leading to that observed dispersion. Conclusions We have presented a strategy capable of providing a structured analysis of effects operating on test results and reconciling experimental and other data with the information requirements of the GUM approach. The initial analysis technique is simple, visual, readily understood by analysts and encourages comprehensive identification of major influences on the measurement. The reconciliation approach is justified by comparison with the ISO GUM principles, and it is shown that the two approaches are equivalent given representative experimental studies. The procedure permits the effective use of any type of analytical data, provided only that the ranges of relevant influence parameters involved in obtaining the data can be established with reasonable confidence. The chief disadvantage is that the contributions to the uncertainty estimate will often combine the effects of many factors, whose individual contributions are not then available for use in method optimisation. This disadvantage is offset to an extent by the production, during the process, of a comprehensive list of influence parameters, which will naturally assist in planning additional optimisation work. Further, if an overall estimate is all that is required, it is a considerable advantage to avoid laborious study of many effects. The implications of the reconciliation approach for the use of holistic ( top-down ) approaches to uncertainty estimation, such as interlaboratory study, have been discussed briefly. It is argued that, while proper planning of a holistic study can guarantee adequate accounting for all effects in a small number of experiments, the practice of random selection of analysts, laboratories, etc., corresponds to selection of a representative sample for many major influence parameters. The dispersion of results observed in such experiments is, therefore, equivalent to ISO GUM uncertainty subject to additional contributions from parameters held constant during the studies. Production of this paper was supported under contract with the Department of Trade and Industry as part of the National Measurement System Valid Analytical Measurement Programme. References 1 Sargent, M., Anal. Proc., 1995, 32, Ellison, S. L. R., Williams, A., and Wegscheider, W., Anal. Chem., 1995, 69, 607A. 3 ISO, Guide to the Expression of Uncertainty in Measurement, International Standards Organization, Geneva, EURACHEM Guide: Quantifying Uncertainty in Analytical Measurement, Laboratory of the Government Chemist, London, Giacomo, P., Metrologia, 1981, 17, Analytical Methods Committee, Analyst, 1995, 120, Pueyo, M., Obiols, J., and Vilalta, E., Anal. Commun., 1996, 33, Williams, A., Anal. Proc., 1993, 30, Ellison, S. L. R., in Advanced Mathematical Tools in Metrology III, ed. Ciarlini, P., Cox, M. G., Pavese, F., and Richter, D., World Scientific, Singapore, 1997, pp Horwitz, L. W., Pure Appl. Chem., 1988, 60, AOC recommendation, J. Assoc. Off. Anal. Chem., 1989, 72, ISO 5725: 1994, Accuracy (Trueness and Precision) of Measurement Methods and Results, International Standards Organization Geneva, Ellison, S. L. R., and Williams, A., Accred. Qual. Assur., 1998, 3, 6.

6 1392 Analyst, June 1998, Vol ISO : 1993, Total Quality Management Part 2. Guidelines for Quality Improvement, International Standards Organization, Geneva, Kragten, J., Analyst, 1994, 119, Ellison, S. L. R., and Barwick, V. J., Accred. Qual. Assur., 1998, 3, Peiser, H. S., De Bievre, P., Goodman, P. R., and Ku, H. H., Accred. Qual. Assur., 1996, 1, 67. Paper 7/06946D Received September 25, 1997 Accepted March