No. 4 (009) Page: 1 of 45 Date: June 009 CONTENTS 1. Introduction. Validation of a chemical method 3. The validation procedure 3.1. Plan 3.. Specificity 3.3. Standard curve 3.4. Precision 3.4.1. Quantitative determinations 3.4.. Qualitative determinations 3.5. Trueness 3.5.1. Analyses for which certified reference materials or other reference materials are available 3.5.. Analyses for which reference methods are available 3.5.3. Analyses for which organised proficiency testing schemes are available 3.5.4. Analyses for which certified reference materials, reference methods or proficiency testing schemes are not available 3.6. Concentration and measurement range 3.7. Limit of detection 3.7.1. Quantitative determinations 3.7.. Qualitative determinations 3.8. Limit of quantification 3.9. Ruggedness 3.10. Evaluation of validation results 4. Documentation 5. Monitoring 5.1. Continuous monitoring 5.. Monitoring following changes in the procedure 6. Example full internal method validation 6.1. Specificity 6.. Standard curve 6.3. Precision 6.4. Trueness 6.5. Measurement range 6.6. Limit of detection 6.7. Limit of quantification 6.8. Ruggedness 7. Word lists 8. References NORDIC COMMITTEE ON FOOD ANALYSIS
No. 4 (009) Page: of 45 Date: June 009 FOREWORD The first version of this procedure was issued in 1996. It was elaborated by a working group established by the Nordic Committee on Food Analysis (NMKL) at its annual meeting on the Faroe Islands in 1995. The procedure was revised in 005. In 006 NMKL arranged courses in the procedure within all the Nordic countries. The assessment reports of the courses were positive. It was notified a need for inclusion of a practical example in the procedure. This has been taking into account in this 3 rd version. The example (chapter 6) is elaborated by the lecturers of the courses: Torben Leth, the National Food Institute, Technical University of Denmark and reviewed by Joakim Engman, National Food Administration, Sweden. Hilde Skaar Norli, NMKL Secretary General, has included the example in the procedure. Torben Leth has elaborated the given example in Excel Spread sheets which are made available for downloading at NMKL s homepage. The following persons have participated in the elaboration of the NMKL Procedure No 4: Kåre Julshamn. NO 3 Per Lea NO 6 Johan Lindeberg SE Lisbeth Lund DK 4 Inge Meyland DK 1, Anders Nilsson SE 1 Elísabet Jona Sólbergsdóttir IS Arngrímur Thorlacius IS 1 Harriet Wallin FI 1, 5 Torben Leth DK 7 Joakim Engman SE 7 1: Participated in 1996 : Participated in 004 3: Chairperson in 1996 4: Observer in 1996 5: Chairperson in 004 6: Secretary/author in 004 7: Held courses in the procedure in 006 and elaborated a practical example for the 009 version. The procedure is available at the web shop at www.nmkl.org and from: NMKL General Secretariat National Veterinary Institute, PB 750 Centre, N-0106 OSLO, NORWAY Tel.: (+47) 3 1 6 50 E-mail: nmkl@vetinst.no NMKL invites all readers and users of the procedure to submit comments and views on its contents to the General Secretariat of NMKL (address above).
No. 4 (009) Page: 3 of 45 Date: June 009 1. INTRODUCTION Authorities who have chemical analyses carried out in connection with the control of food, drinking water or animal feed, scientists performing analyses in conjunction with research projects or assignments, and individual chemical laboratories all need to know whether an analysis result is correct or good. In connection with accreditation, it is an obvious requirement that the method of analysis is up to standard. It is important to be aware of the fact that the best way of ensuring that a laboratory s analyses satisfy all relevant requirements, is to participate in a proficiency testing (PT) scheme. However, the number of available and potential analysis methods is ever increasing, and for different reasons, it may not always be possible to participate in a PT. This may for example be due to the fact that the analysis is so specialised that not enough laboratories perform it. The users of this procedure will represent laboratories of different sizes and with varying resources. In some cases, it may be appropriate to adjust the suggested criteria to each individual laboratory s activities. This in turn means that the laboratories are free to change some of the criteria. Such changes must be documented by attachments to the relevant laboratory s validation procedure. Although this procedure deals with the validation of a method, it is really about validating an analytical unit. This entails that elements such as premises, instruments, equipment, human resources, defined concentration ranges, and defined matrices are implicitly included in the validation. Consequently, it may be necessary to repeat the validation process should changes occur in one or more of these elements. The concept of measurement uncertainty is not given an in-depth treatment in this procedure. This is covered in NMKL Procedure No. 5: Estimation and expression of measurement uncertainty in chemical analysis (005).. VALIDATION OF A CHEMICAL METHOD Validation of a method means examination and determination of the parameters of the method. This can either be done by a single laboratory (internal validation), or by several laboratories (collaborative study). Verification of a method is an examination of a single laboratory s ability to perform the analysis in accordance with the method parameters established in the validation. Key method parameters are (if relevant to the current method): Field of application Trueness Precision HorRat value (=RSD Obtained /RSD Predicted ) Limit of quantification
No. 4 (009) Page: 4 of 45 Date: June 009 As will appear from the table below, the extent of the verification performed locally in each laboratory depends on how thoroughly the method has been validated externally: a thorough external validation simplifies the internal verification. It is possible to divide methods of analysis into 6 categories, depending on the degree of validation that has been documented for the method: Degree of external validation 1 The method has been externally validated in a collaborative study. The method has been externally validated in a collaborative study, but is used on a new matrix or with a new instrument. 3 The method is well established but has not been externally validated in a collaborative study. 4 The method has been published in scientific literature, and states important performance characteristics. 5 The method has been published in scientific literature, but lacks important performance characteristics. Recommended internal verification Verification of trueness and precision. Verification of trueness and precision, and possibly also limit of quantification. Verification, and possibly a more extensive internal validation. Verification, and possibly a more extensive internal validation. Full internal validation. 6 The method is developed internally. Full internal validation. It should be emphasized that some methods, although issued by standardization bodies, have not been validated through collaborative studies. The method has been externally validated in an collaborative study means that it has been through a collaborative study in accordance with internationally accepted guidelines, e.g. ISO 575 (1994-1998), Horwitz (1995) or NMKL Protocol No. 1 (005), and has given acceptable results. Furthermore, a collaborative study report containing information about the method parameters which are relevant for the current method, must be available. It is becoming increasingly common to render the results of the collaborative studies in the method text itself, or to publish it as a separate report which is referenced in the method description. By verification is understood that the laboratory, prior to using the method in routine assignments, tests and documents its competence in using the method. This means that the laboratory tests and documents that it is able to obtain results for trueness and precision corresponding to those which are given in the collaborative study. Above all, the laboratory must demonstrate that the method is suitable for solving the analytical task in question.
No. 4 (009) Page: 5 of 45 Date: June 009 By a more extensive validation is understood that the laboratory examines and documents, fully or partly, the method s characteristics (see section 3) before taking it into routine use. By full internal validation is understood an evaluation of all relevant method characteristics (see section 3). The extent of the validation is determined by the purpose of the method. Methods that are to be used in public food control requires full validation in accordance with point 1 in the table, whereas methods that are to be used internally for process management, may only need a simple verification. For methods that are to be used for assignments for internal or external customers, the laboratory or the customer is free to determine how strict the verification should be. Methods that are to be used for concentrations close to the detection limit, often require a more extensive validation on lower levels than methods that are to be used for concentrations far from the detection limit. The extent of the validation also depends on whether it is a quantitative or a qualitative analysis, and which principles of analysis are used. Internal validation or verification must always be performed for a new analytical method for routine analyses before it is taken into use. This work must be repeated partly or in full if the result of the first validation or verification makes it necessary to modify the method. The applicable concentration range and sample matrix is an important part of the validation or verification of the analysis method. It will often save both time and effort to let the laboratory s needs determine the area of application, rather than the limitations of the methods. It is also important that the validation and verification report describe which sample matrices have been used. The function of some methods may for instance be dependent of the matrix. In such cases, it is very important to ensure that the entire stated area of application is included by the validation. The validation and verification work shall not commence until the development of the method in considered concluded. It is not possible to make any changes to the method during the validation or verification process. If the method is modified, the whole validation or verification process must be repeated.
No. 4 (009) Page: 6 of 45 Date: June 009 Validation Verification 3.. Specificity 3.3. Calibration, standard curve 3.4. Precision 3.5. Trueness 3.6. Concentration range, measurement range 3.7. Limit of Detection 3.8. Limit of Quantification 3.9. Ruggedness 3.10. Evaluation
No. 4 (009) Page: 7 of 45 Date: June 009 3. THE VALIDATION PROCEDURE The flow chart on the previous page indicates which elements are included in a validation and a verification process. The points 3.. 3.9. describe the elements which must be examined in a full internal validation of an analysis method. If, in an internal validation, some of these points are omitted, the reason for this must be documented. The points 3.4 (precision) and 3.5 (trueness) describe the elements which must be examined in a verification. This can be supplemented with elements from 3.7 (limit of quantification). 3.1. PLAN Before the experimental part of the validation or verification starts, a plan for the work must be drawn up. The plan shall include an overview of what the method is to be used for, and must take the following factors into consideration: The needs of the customer o The requirements of the control authorities What is analytically possible Internal conditions in the laboratory o Working environment legislation o Equipment o Resources The validation plan should answer the following questions, in the extent that they are relevant to the method: QUESTION For what purpose is the method to be used? COMMENT Example: Public food control, continuous production control in an industrial company. Does the choice of analytical principle influence the requirements for reproducibility? Is a qualitative or quantitative result required? In which chemical forms does the analyte occur: - Bound or free? - In different chemical compounds? This consideration affects specificity, limit of detection, limit of quantification, etc. This has a bearing on the choice of sample matrices selected for the validation of recovery tests. What is the area of application of the
No. 4 (009) Page: 8 of 45 Date: June 009 QUESTION COMMENT method? Is there a risk of interference from the sample matrix or other analytes? How much sample is available and is the sample homogenous? In which concentration ranges will the method be used: Near the quantification limit or at higher concentrations? How will trueness be ensured? Which environmental requirements must be fulfilled: - Considerations regarding personal safety? - Dangerous environmental discharges? What are the financial limitations? Affects the requirements for selectivity and specificity. Has a bearing on the requirements for reproducibility, repeatability or specificity. Affects requirements for linearity, limit of detection, limit of quantification, reproducibility and repeatability. Compare with relevant reference materials or participate in proficiency testing schemes. Surrounding conditions may influence the analysis through extreme temperatures or particles in the air. This consideration affects the requirements for ruggedness, and thus indirectly, requirements for personnel, equipment, reagents and premises. For commercial laboratory activity: If future use of the method will generate less income than the cost of validating it, it is hardly sensible to take it into use. The validation and verification plan shall describe which validation elements that are to be evaluated, and in which order this will take place. It must also state the requirements which the laboratory establishes for the various points in the plan. If the customer does not require specifications related to repeatability or reproducability, the laboratory itself shall state analytically reasonable requirements for the method. 3.. SPECIFICITY Definition: Specificity is the ability of an analytical method to distinguish the analyte to be determined from other substances present in the sample. A blank sample and one or more samples to which a known amount of the analyte has been added, are analysed to check that there are no interferences with the analyte from other compounds in the sample, from degradation products, metabolites or know additives. In some
No. 4 (009) Page: 9 of 45 Date: June 009 cases, for example in the analysis of pesticides, a more concentrated extract of the blank may be analysed in order to demonstrate that no signals occur. The specificity of a method is checked by comparing it with other methods based on other principles of analysis. Specificity can also be examined by carrying out determinations in the presence of substances suspected of interfering with the analyte. However, the analyst must be aware of the fact that the analyte may be present in the sample in more than one chemical form. By experience, analysts are often familiar with what kind of interferences that are to be expected for a specific method, such as spectral interferences in connection with ICP analyses or identical retention times for several substances when using chromatography. Such problems may have an unfortunate impact on the analysis result, and these methods therefore require a more extensive examination of the specificity than techniques that are associated with none or only a few interferences. 3.3. STANDARD CURVE Definition: A standard curve is a function which reflects the correlation between the content of an analyte in a sample, and the resulting measurement response. The standard curve must be determined by a certain number of relevant measurement points, depending on method and product. In the following, a linear correlation is assumed, at least in the relevant measurement range. The analyses are performed on reference samples with a known content or blank samples to which a known concentration of the analyte is added. It is of the utmost importance that the points for determination of the standard curve are within a measurement range which is relevant for how the method is to be used in practice. The experiment should be repeated at least once. The results may be presented graphically, and the equation should be given for the linear regression as well as the correlation coefficient for each of these experiments. If the correlation is not linear, the parameters in the relevant curve may be estimated, either using the smallest square method, or by means of numerical methods if the problem cannot be solved analytically. For multivariable calibrations, please refer to Martens and Næs (1989). The linearity cannot be evaluated based only on the size of the correlation, but as recommended by Tiley (1985): Let s1 be the variance for the adjustment error: s = 1 1 ( y ˆ) n y
No. 4 (009) Page: 10 of 45 Date: June 009 where y represents the measured y values, ŷ the estimated values based on the regression equation, and n is the number of point pairs (x,y). The standard deviation for the y s, i.e. the precision of the y s, is given by: s = 1 n 1 y ( y ) The values s F = s 1 s 1 and s are stochastically independent, and is F-distributed with n- and n-1 degrees of freedom (see e.g. Draper and Smith (1981), chapter 1.3.) under the zero hypothesis: H 0 : The correlation between x and y is linear. If the F test leads to rejection of H 0, it may be concluded that the correlation is not linear. Spectrometric methods most often have a linear standardisation area up to a certain concentration. At higher concentrations, the curve deflects towards the concentration axis. Unless the method is used for a very limited range of purposes where the relevant concentration range is already known (and lies within the linear area), the size of the linear area should be examined. It is also necessary to map out in how large a part of this area the method s requirements for precision and trueness are fulfilled. This will give a limitation at the lower end of the standard curve in the form of a quantification limit. An ion-selective electrode often shows a response which is a linear function of the logarithm of the concentration across a large concentration range, but gives an inclination towards the tension axis when the concentration moves towards 0. The limit of quantification can then be defined as the lower end of the linear part of the standard curve. If the analyses yield results outside of the linear area, it is recommended to dilute the samples so that the results fall within the linear area. In some cases it is more appropriate to use a non-linear standard curve; either because it is desirable to go outside the linear area, or due to the usage of a detector with a non-linear response which is difficult to linearise using mathematical functions. One example of this is flame photometric sulphur detector for gas chromatography. In the case of non-linearity, it is also necessary to map out the concentration range in which the correlation between concentration and response applies, and whether the other quality requirements for the method are fulfilled.
No. 4 (009) Page: 11 of 45 Date: June 009 3.4. PRECISION Definition: Precision is the degree of agreement between independent analysis results obtained under specific circumstances. 3.4.1. Quantitative determinations The precision of an analysis depends only on the distribution of the random errors of the analysis, and must not be confused with trueness. Precision is usually expressed as the standard deviation of the analytical results. A small standard deviation means high (or good) precision, and a large standard deviation means low (or poor) precision. The concept of precision is only relevant to quantitative analyses. Precision is a relative concept, totally dependent on the specific conditions mentioned in the definition above: the extremes are repeatability conditions and reproducibility conditions. Definition: Repeatability means that the analysis results are obtained using the analytical method on identical samples in the same laboratory, using the same equipment within a short period of time. Reproducibility means that the results are obtained by using the analytical method on identical samples in different laboratories and using different equipment. Internal reproducibility means that the determination is carried out at different times, by different persons and on different batches of reagents but in the same laboratory. It must be emphasized that the precision is usually dependent on the analysis technique, and quite often also on the concentration of the analyte. In order to estimate the precision, a certain number of parallel samples of the same sample material are analysed. The other conditions (equipment, persons, time etc.) may be in accordance with the requirements for repeatability, reproducibility or something in between. The standard deviation is calculated based on either replicate single determinations, or on duplicate determinations. The standard deviation of single determinations is an estimate of the variance around the average, whereas the standard deviation derived from duplicate determinations is an estimate of the average variations of the difference between two single analysis. Single determinations: The standard deviation is defined as:
No. 4 (009) Page: 1 of 45 Date: June 009 s = n (x i= 1 i x) n 1 which for calculation technical reasons may also be written like this: s = n x i= 1 i nx n 1 where x 1, x,..., x n are the individual determinations, and values. 1 n x = x i is the average of the x i n i= 1 The standard deviation is in practice calculated by means of a thoroughly tested spreadsheet program or statistical program. (When using spreadsheets or calculators; note that some of these calculate the standard deviation with n instead of n-1 in the denominator.) Duplicate determinations: A duplicate determination consists of two measurements; x i and y i. The standard deviation is the average standard deviation for all the pairs. Since the variance based on observations, x and y, is reduced to (x y), the average of n such pairs is: 1 n (x n i= 1 i n (x y ) y ) i i i i= 1 = n and on the standard deviation level (the square root of the variance): s = n (x i= 1 i y ) n i The precision is expressed directly either by: the standard deviation, s
No. 4 (009) Page: 13 of 45 Date: June 009 s the relative standard deviation RSD = 100% x s s the confidence interval x c, x + c n n The constant c is dependent on the confidence level, and is found in a table of the t- distribution with n-1 degrees of freedom. For confidence level 1-α, c is equal to the (1-α/) fractile. c=3 is quite often used. This is approximately equal to a confidence interval of 99% when the number of measurements is 15. c= gives a confidence interval of approximately 95%. (See Appendix : Tables for t-tests. 15 measurements indicates Df = 14, and a confidence interval of 99% corresponds to α = 1% or 0.01, and 1-α/ = 0.995. Df = 14 and a probability of 0.995 gives a c value of.9768 3. A confidence interval of 95% corresponds to α = 5% or 0.05, and 1-α/ = 0.975. Df = 14 and a probability of 0.975 gives a c value of.1448.) The table below contains values for the relative standard deviations which are to be expected for different concentrations. The table is based on a large number of collaborative studies, and is taken from Pocklington (1990). A reproducibility standard deviation no higher than twice the value stated in column, is considered acceptable according to Pocklington. However, this is not a universal truth established once and for all. For example, EU requirements stipulate that for analytical methods for lead, cadmium and mercury in connection with public control, this value should not be higher than 1.5 (EU (001)). 1 3 4 5 Concentration RSD % (Reproducibility) Acceptable reproducibility RSD % (Repeatability) Acceptable repeatability 10-1.8 5.7 1.9 3.8 10-4.0 8.0.7 5.3 10-3 5.6 11 3.8 7.5 10-4 8.0 16 5.3 11 10-5 11 3 7.5 15 10-6 16 3 11 1 10-7 45 15 30 For concentrations below 10-7 Analyst, 000). it is recommended to set RSD% to % (M.Thompson, Experience has shown that the values for the repeatability standard deviations are 1/-/3 of the reproducibility standard deviations (see column 4). (The numbers in the table are rounded off to significant figures. This is the reason why the numbers in column 3, 4 and 5 aren t always exactly the results one would obtain based on the numbers in column.) Instead of using the table, it is possible to enter the actual values in the formula on which the table is based; i.e. the HorRat value (Horwitz Ratio):
No. 4 (009) Page: 14 of 45 Date: June 009 HorRat = RSD RSD Obtained Pr edicted RSD Obtained is the estimated relative reproducibility standard deviation from a collaborative study, whereas RSD Predicted is an empirical value based on a large number (pr. July 004, approx. 10 000) of collaborative studies: RSD Predicted = (1 0,5logC) = C 0,5log C 0,1505 where C is the mean value of the measured mass concentrations (measured in mg/g, g/kg, µg/g or a similar unit). Using the formula instead of the table, makes it possible to avoid problems in the marginal cases: the requirement that HorRat should give approx. the same result if the measurement values are close to 9 10-3 or 1 10 -, i.e. around 0.009 or 0.010. If the table is used non-discriminatorily, acceptable relative reproducibility standard deviation is 11 for 9 10-3, and 8.0 for 1 10 -. But if we enter C = 9 10-3 and C = 1 10 -, the requirements will be 8.1 and 8.0, respectively. According to Horwitz (1995), his formula is only based on mass concentrations, and can therefore not without further consideration be used in connection with other types of measurement. 3.4.. Qualitative determinations Qualitative determinations are usually binary, i.e. they give one of two possible results such as yes/no, 1/0, over/under a threshold concentration, etc. The precision of a qualitative analysis may be expressed as the percentage share of false positives and false negatives: % % false positives = false negatives = Numberof false positives 100% Sumof known negatives Numberof false negatives 100% Sumof known positives 3.5. TRUENESS Definition: Trueness is the degree of agreement between a sample s true content of a specific analyte and the result of the analysis. The result can be either a single value or a function of several single values. Such a function is usually the mean value, but in some cases it may be appropriate to use median values or other forms of modified mean values.
No. 4 (009) Page: 15 of 45 Date: June 009 It is important to acknowledge the fact that a sample s true content of an analyte is always unknown. In order to evaluate the trueness of a method, it is therefore necessary to depend on accepted results such as: a certified content of a reference material results obtained using a validated method (provided that sample(s) with a known content have been included in the collaborative study) results from a proficiency testing (PT) In all three of these cases results from several laboratories, analysts and instruments are included, thus minimising the effect of individual errors. Trueness is an important factor in the evaluation of all types of quantitative analytical methods. This also applies to the methods which define their own area of application. Furthermore, obtaining results which are comparable to the results obtained by other laboratories using the same method, will always be of great significance. Some analytical methods define their own measuring result, and the area of application is then implicit. Examples: Kjeldahl-N (defined as the amount of Nitrogen measured in a Kjeldahl analysis) and amount of solids (drying at 105 C, vacuum drying at 70 C or freeze drying). For both trueness and precision, the requirements will be dependent on the concentration level and purpose of the analysis. For example, in trace analysis a deviation of over 10% may be acceptable both for precision and trueness, whereas such deviations would be totally unacceptable when measuring Kjeldahl-N or solids. 3.5.1. Analyses for which certified reference materials or other reference materials are available NMKL Procedure No. 9 (0071) includes a more detailed description of analysis results from certified reference materials. The trueness of a method can be examined by analysing a certified reference material. If a certified reference material is not available, use another reference material or a material which has been prepared in the laboratory ( in-house standard ) with a known content of the analyte. The content of the analyte in such a control material must be thoroughly examined, preferably using two or more analytical methods based on different physical and/or chemical principles, and organised so that the analyses are performed at more than one laboratory. An in-house standard should, if possible, be calibrated against a certified reference material. Failing to assess the analyte concentration of an in-house standard accurately is a serious breach of the quality assurance procedures of the laboratory. A certified reference material or an in-house standard should be based on a matrix, and have an analyte concentration which covers that of the samples for analysis as far as possible. It is important to be aware of the fact that certified reference materials or other reference materials can only be used to evaluate the analytical method on the concentration level which has been examined.
No. 4 (009) Page: 16 of 45 Date: June 009 Certified materials and other control samples are not always representative of a typical sample in a food control laboratory. Certified materials can be: easier to handle easier to extract easier to ash more homogenous and have fewer interferences than real food samples. Due to this, the results obtained for reference samples are often better than the results obtained with real samples. This may give a false sense of security, and it is not possible to use results from analyses of reference materials alone as proof of the trueness of an analysis of an unknown food sample. Analysis of the reference materials alone can not verify the trueness of the method. These analyses must be supplemented with other criteria of reliability, such as recovery tests. 3.5.. Analyses for which reference methods are available Definition: In this context, a reference method is an analytical method which has been studied in a collaborative study with good results, and which has an acceptable trueness. The trueness of the analytical method can be examined by analysing the same sample both with the relevant method for internal validation, and with a reference method. If the laboratory has not yet adopted the reference method, there is little point in introducing it only to evaluate a new analytical method. In such cases, it is recommended to send the samples to another laboratory, preferably one that is accredited for the relevant method, and have the samples analysed there. Whether there is a significant difference between the results obtained for the reference method and those obtained for the method which is to be validated internally, is determined by performing a t test (Appendix 1), or alternatively a variance analysis. 3.5.3. Analyses for which organised PT-schemes are available The trueness of the analytical method is examined by participating in a PT-scheme for samples equivalent to the type of sample the method will be used for. The documented trueness only applies to the concentration range and the matrices which are included in the PT-scheme. If such testings do not exist, smaller comparisons with a few laboratories, even only one other laboratory, can in some cases provide valuable information. It is assumed that the laboratory organising the PT is able to document its competence as specified in ISO/IEC Guide 43 (1996) or Thompson et al (006).
No. 4 (009) Page: 17 of 45 Date: June 009 3.5.4. Analyses for which certified reference materials, reference methods or proficiency testing schemes are not available Definition: The concept of recovery actually covers two concepts (Burns et al (00)): Recovery factor, which is the yield of an up-concentration or extraction step in an analytical process divided by the amount of analyte in the original sample. Apparent recovery, which is the observed value of an analytical method using a calibration function divided by the reference value. If neither certified reference materials, nor reference methods nor PTs are available, it is necessary to make use of other methods to ensure the trueness of the analyses. Furthermore, it is paramount that the laboratory takes additional quality assurance measures, and cooperates with a reputable laboratory for the exchange of samples and results, as well as relevant experiences. Recovery is one suitable method which may be used when the resources mentioned in the title are not available. As demonstrated by Burns et. al. (00), this concept, however, is not always used in the same manner by everyone. In the above-mentioned article, there is a distinction between recovery and apparent recovery. Recovery (represented by R) is used in connection with a upconcentration or extraction step in an analytical method: Q R = Q Found Original where QFound is the amount of the analyte recovered after processing the sample, and QOriginal is the known, original amount. If standard addition or spiking is used, the recovery is calculated according to the following formula: Q R = Found Q Q Original sample Spiked where Q Found is the amount of analyte measured (which contains the original amount of analyte plus the added amount), Q Originalsample is the amount of analyte measured in the original sample, and Q Spiked is the added amount of analyte. When recovery is used in connection with an analytical process in which a calibration curve is included, it is defined like this: R'= x x Ref
No. 4 (009) Page: 18 of 45 Date: June 009 where x is the read value, and x Ref is a reference value which comes from a certified reference material. When spiking is used, the following definition is applied: R' = x (O+ S) x - x (S) (O) where x (S) is the amount added for spiking, x (O) is the measured amount in the original material, and x (O+S) is the measured amount in the spiked sample. The reason for distinguishing between R and R, is that the calibration line can have both an additive and a multiplicative systematic error: x = a + bx Ref where a and/or b are different from 0. Thus, R can in principle become larger than 100%, and it can be equal to 100% without that being any guarantee of the method working perfectly. Therefore, it is essential that the method in such cases is validated either by using several reference materials, or by using a suitable interval for the amount of the analyte. An analytical method with a linear calibration curve does not require a recovery (R) of 100% in an up-concentration or extraction procedure. It is required that the unknown sample and the calibration samples have the same recovery so that (R ) becomes 100%. The technique is especially suitable for unstable analytes, or in cases where only a few analyses are to be performed. It is important that recovery tests are carried out in the relevant concentration range for analysis. If in future the method will be used on several levels, recovery tests should be carried out on a selection of these levels. The great advantage of using recovery tests is that the matrix is representative of authentic samples. The technique can be used on all analytes and on most sample types, provided that the analyte exists as a stable synthetic chemical compound in the laboratory. The greatest limitation of this method is the fact that the analyte in the natural sample can be strongly physically or chemically bound to the matrix, as would normally not be the case for the added analyte. This means that it is perfectly possible to obtain a high recovery percentage for the added analyte, without being able to obtain a complete determination of the naturally present analyte. If the recovery result lies within the area of 80%-110%, it is often sufficient to analyse the series (i.e. at least 5 samples with addition and at least 5 samples without addition) 3 times. The analyses should be carried out within a limited time span. A general rule is that the lower the analyte concentration, the more determinations must be carried out in order to obtain satisfactory estimates for the yield. The reason for this is that the random errors measured as relative standard deviations, increase when the analyte concentration decreases. To examine whether the established recovery percentage is significantly different from 100%, it is possible to use a simple t test (see appendix 1).
No. 4 (009) Page: 19 of 45 Date: June 009 3.6. CONCENTRATION AND MEASUREMENT RANGE Definition: The measurement range is the range in which the method is validated and which gives an acceptable trueness and precision. All methods have a limited sensitivity which restricts the concentration range the method is suitable for. The lower limit for reliable quantification is called limit of quantification, and is dealt with in section 3.8. 3.7. LIMIT OF DETECTION 3.7.1. Quantitative determinations Definition: The limit of detection is the amount of an analyte corresponding to the lowest measurement signal which with a closely defined confidence may be interpreted as indicating that the analyte is present in the sample, but without allowing exact quantification. The detection limit is based on the standard deviation for natural samples not containing the analyte, blank samples or natural samples, or standards with a very low content of the analyte. By low content is understood a content which is as close as possible to the expected limit of detection. Procedure: Analyse a certain number (n) of blank samples (a least 0), and define the detection limit as a constant c multiplied with the standard deviation for the average concentration of the blank samples. The constant c is found in a table containing the t- distribution with the degrees of freedom equal to n-1 and level 1-α. The most commonly used value is α=1%, i.e. α=0.01. For α=0.01 and n=0, c=3 is a frequently used approximation to the value from the table. The detection limit can also be determined by means of the standard deviation for standard solutions with extremely low concentrations or blank samples, by adding small amounts of standard. For some analytical methods, it is not possible to obtain a definite signal for the blank sample. As a last resort in such cases, it is possible to try and magnify the instrumental interference, and specify the limit of detection as c times the standard deviation of the interference. Blank sample extract must be used when magnifying instrumental interference. This extract should be significantly more concentrated than normal, in order to establish that no components are interfering with the determination of the analyte. The limit of detection is particularly important in all trace analyses, but less significant to the evaluation of methods used for the determination of the main components of foods.
No. 4 (009) Page: 0 of 45 Date: June 009 3.7.. Qualitative determinations Definition: The limit of detection is the threshold concentration below which positive identification is unreliable according to the established requirements for reliability. It is recommended to firstly examine in what concentration range the method yields correct results. This is done by analysing a series of samples comprising a blank sample and samples which contain different concentrations of the analyte. It is recommended to carry out at least 10 parallels on each concentration level. Draw up a response curve by plotting the share of positive results on the Y axis and the concentration on the X axis. It is then possible to read from the curve the threshold concentration at which the method starts to become unreliable. In the example below, the reliability of the detection method becomes less than 100% at concentrations below 100 µg/g. Concentration (µg/g) n Positive Negative Positive/negative (%) 5 10 0 10 0 50 10 1 9 11 75 10 5 5 50 100 10 10 0 00 10 10 0 3.8. LIMIT OF QUANTIFICATION Definition: The lowest amount of an analyte which can be determined quantitatively with a closely defined confidence. Analyse a number of blank samples. The quantification limit is 10 times the standard deviation for the average of the blank sample. Here, too, it is possible to add small amounts of standard, as described in section 3.7.1. How to use instrumental interference as a basis for determining the quantification limit is described in section 3.7.1. 3.9. RUGGEDNESS Definition: The sensitivity of an analytical method to minor deviations in the experimental conditions of the method. A method which is not influenced by such minor deviations, is said to be rugged when it comes to these experimental conditions. Some form of ruggedness testing should be included in the evaluation of analytical methods which are elaborated in a laboratory. In relevant literature it is recommended to perform ruggedness tests before a collaborative study is started. The nature of the analytical method in
No. 4 (009) Page: 1 of 45 Date: June 009 question will determine which parameters need to be tested. The most frequently tested parameters, which may be critical to an analytical method, are: the composition of the samples the batch of chemicals ph extraction time temperature pressure liquid flow rate volatility Blank samples can be used for ruggedness testing as they will reveal effects caused by the matrix or the chemical batch. Information from the ruggedness test can be used to specify the conditions under which a method should be used. Please refer to Steiner (1975) for more detailed information on ruggedness tests. 3.10. EVALUATION OF VALIDATION RESULTS It is important to continuously compare what actually takes place during the validation and verification process to the original plan. If the desired specifications for linearity or trueness are not fulfilled, the method must be modified or changed, and in both cases, the validation and verification work must be repeated. 4. DOCUMENTATION OF VALIDATION AND VERIFICATION The documentation can be divided into 5 different categories: 1. Planning and preparations (see section 3.1).. Documentation of primary data for the experiments (see section 3., 3.3). 3. Documentation of the evaluation of validation data (see section 3.4-3.9). 4. Comparing and evaluating validation results (see section 3.10). 5. Validation or verification report which is to be written after the work has been completed. This report shall contain: All raw data or a reference to where the raw data may be found. An accurate specification of which characteristics have been examined. If not all characteristics have been examined, the reason for this must be given in the report. Obtained results and how these have been calculated. It is especially important to include which matrices and at which concentrations there is a specific precision, trueness, and specific detection limits and quantification limits. An account of how to evaluate obtained results in relation to the validation plan or verification plan.
No. 4 (009) Page: of 45 Date: June 009 An unambiguous conclusion concerning which analytical tasks the examined method is suitable for. If the method is found to be unsuitable for some matrices or concentrations, this must be evident in the report. Accredited laboratories shall follow the rules for documentation and archiving in section 4.3. of ISO/IEC 1705 (005). All reports must be archived together with their corresponding plan, and must be retained for as long as the corresponding method is archived. These reports and reports related to periodical monitoring shall be made available to all analysts who use the method. It is essential to record exactly how and on which materials the repeatability or internal reproducibility (reference material, control material, authentic samples or synthetic solutions) has been determined. If the analytical method being studied is to be used for a large concentration range, the analyte precision shall be estimated on several concentration levels, usually low, medium and high. It is recommended that the estimations of precision is repeated in part or in full during the validation work. 5. MONITORING When a validated/verified method is taken into routine use in a laboratory, it is important that a competent analyst is made responsible for monitoring that the method continuously performs in accordance with the results obtained in the validation/verification. Below is an example of guidelines for how this monitoring may be carried out. 5.1. CONTINUOUS MONITORING The trueness and precision of the method must be monitored continuously. The obtained results must be in accordance with the results of the validation or verification. How often such control should be carried out, will be determined by the analyst responsible and must be thoroughly documented, for example in the internal analytical user manuals. See also NMKL Procedure No. 3 (1996). 5.. MONITORING FOLLOWING CHANGES IN THE PROCEDURE Changes in the experimental conditions for a method can cause a modification of the function of the method. In such cases, it may be necessary to carry out a new, complete validation. Minor modifications of the method If the method has been modified to a smaller or larger degree, it must be ensured that after the modification, it gives equally good or better results when it comes to detection limit, specificity, trueness and precision for all relevant matrices.
No. 4 (009) Page: 3 of 45 Date: June 009 Use of new matrix If the method is to be used on a matrix which was not included in the validation or verification, the usability of the new matrix must be ensured by checking specificity, trueness and precision. New chemicals If chemicals from a new manufacturer is to be taken into use, or large batch variations are expected, it must be checked that detection limit and sensitivity remain unchanged. New instruments When new instruments are taken into use, the measurement range, linearity, quantification limit and precision must be checked. New premises If the analyses are moved to new premises and consequently new analytes, matrices and instruments are taken into use, the blank value, quantification limit, linearity and precision must be checked. New analyst If a new analyst will be working with a method, it must be ensured that the person in question is sufficiently well versed for the task. This may be done by checking the quantification limit, trueness and precision. The method has not been used for a long time If the method has not been used for a long time, the quantification limit, trueness and precision must be checked. 6. EXAMPLE OF FULL INTERNAL METHOD VALIDATION This chapter describes how to do an extensive full internal method validation. The example used is a new developed method for the determination of vitamin C (ascorbic acid). The method principle is extraction, HPLC with ion pair chromatography on a C18 column, and UV detection. The method is intended to be applicable to all types of food and dietary supplements, and is expected to yield results on the same level as an older method. The method has in its entirety been developed within one laboratory. Therefore, it is necessary to perform a full internal method validation. 6.1. SPECIFICITY (as described in 3..) A comparison between a new and old method on 3 foods including all relevant matrices has been carried out. No interfering peaks have been observed in the HPLC chromatograms for any foods. The results are given in the table below. The calculations is carried out in Excel spreadsheet and is available as annex 1 to be downloaded at www.nmkl.org under Download Excel spread sheet.
No. 4 (009) Page: 4 of 45 Date: June 009 Table 6.1 Results for vitamin C analysed by old and new method, respectively Vitamin C in foods and dietary supplements mg/100 g Sample Old method New method Mean Difference Relative diff. 1 91 86 (91+ 86)/ = 88.5 (91-86) = 5 (5 100)/88.5 = 5.65 55 59 57-4 -7.0 3 16 16 16 0 0.00 4 1 15 13.5-3 -. 5 41 43 4 - -4.76 6 54 60 57-6 -10.53 7 7 9 8 - -7.14 8 35 41 38-6 -15.79 9 55 57 56 - -3.57 10 55 56 55.5-1 -1.80 11 8 7 7.5 1 3.64 1 5 5 5 0 0.00 13 54 53 53.5 1 1.87 14 5 19 6 7.7 15 7.3 8 7.65-0.7-9.15 16 63 55 59 8 13.56 17 58 57 57.5 1 1.74 18 58 58 58 0 0.00 19 58 57 57.5 1 1.74 0 56 6 59-6 -10.17 1 66 64 65 3.08 93 96 94.5-3 -3.17 3 46 50 48-4 -8.33 The mean of the difference -0.64-1.96 Standard deviation 3.66 9.9 In order to learn if there are significant differences in the results between old and new method (the shaded columns in table 6.1) paired t-test can be used. Figure 6.1 and 6. show how to use Excel (version 003 and 007), for evaluating the data.
No. 4 (009) Page: 5 of 45 Date: June 009 Figure 6.1 How to use the statistics tool in Excel 003 1. At the menu choose Tools and then Add-Ins.. Mark Analysis ToolPak, choose OK 3. At the menu choose Tools and then Data analysis at the bottom
No. 4 (009) Page: 6 of 45 Date: June 009 Figure 6. How to use the statistics tool in Excel 007 1. Press the Office key. Choose Excel Options at the bottom of the window. 3. Choose Add-Ins 4. The list for Manage: shall be Excel Add-ins, press Go... 5. Mark Analysis ToolPak, choose OK 6. If choosing the menu Data, then Data analysis is farthest to the right
No. 4 (009) Page: 7 of 45 Date: June 009 Then scroll down the menu under data analysis for the T-test: Paired two sample for means, as shown below. Choose the column shown in table 6.1 for the old methods as variable 1 and the result for the new method as variable.set hypothesized mean difference to 0. Then automatically you will get a similar table as given below. (Please note that the number of decimals given do not correspond to the accuracy of the methods). T-test: Mean of two paired ranges New Old method method Mean (average of the results of the 3 samples) 46.886087 47.5173913 Variance 503.3987747 495.645059 Number oberservations 3 3 Pearson correlation 0.98660336 Hypothesis of the difference of the mean 0 Degrees of freedom, df t-stat -0.83690087 P(T<=t) one-tailed 0.0581841 t-critical one-tailed 1.717144187 P(T<=t) two-tailed 0.41163681 t-critical two-tailed.07387594
No. 4 (009) Page: 8 of 45 Date: June 009 The results of the t-tests shows that there are no statistical difference between the results [T t]. The difference between the two methods has been plotted against the average content, in order to check for a systematic correlation (see figure below). Comparison of new and old method The difference between the methods mg/100 g 10 8 6 4 0 - -4-6 -8-10 0 0 40 60 80 100 Vitamin C mg/100 g d+sd Mean diff. -0.64 d-sd No correlation between the difference and the content can be seen. All points, except one (in the figure above) are within ± sd. A prerequisite for calculating the absolute numbers, is that the content is approximately the same in all sample pairs. This is not quite the case in this example, therefore, calculating the relative difference between the results of the two methods is more correct. 9.91 For the relative difference, a 95 % confidence interval is % = 4.% The result of these calculations, is an average difference of -1.96% with a standard deviation of 9.9 and a 95% confidence interval for the average difference of 4.. This means that the difference, -1,96% ± 4,%, is not significantly different from 0, and consequently, the methods do not yield significantly different results. 6.. STANDARD CURVE (as described in 3.3) The standard curve should cover the entire relevant concentration range with at least 5-6 points. Every standard solution must be injected at least twice, giving a total of repetitions. In this case, the relevant concentration range is from.5 to 100 µg vitamin C/ml, and 6
No. 4 (009) Page: 9 of 45 Date: June 009 standard solutions have been prepared. The table below contains the relevant concentrations, areas, the difference between the parallels and the square of the difference. Further a regression analysis, as reproduced below is carried out. For the Excel spread sheet, see annex at the www.nmkl.org under Download Excel spread sheet. Table 6. Standard curve µg/ml Area a-b (a-b).5 903 131 17161.5 9189 5 18748 11 158884 5 18616 10 357074 135 175565 10 355749 5 91537-564 6574096 5 917891 50 180777-4546 066793444 50 1853189 100 3604581-3935 10847145 100 3637516 Sum 3161113435 s = 6346119.6 For making a regression analysis in Excel, see figure 6.1 or 6. depending on version. Choose Data analysis, Regression. Press OK, and the following appears:
No. 4 (009) Page: 30 of 45 Date: June 009 For the Y-range include the areas (including the cell with the text Area ) and for the X-range set in the concentration (including the cell with the text µg/ml ). Tick: Label, Confidence level, Residuals Residual plot Standardized residuals Line fit plots Then the data as given in the table and figures below are automatically calculated for you. Summary (Regressions statistic) Multiple R 0.999935545 R-square 0.999871093 Adjusted R-square 0.99985803 Standard error 15484.775 Observations 1 Variance analysis ANOVA Significance F df SS MS F Regression 1 1.86E+13 1.8597E+13 77565.49 8.75995E-1 Residuals 10.398E+09 39761309 Total 11 1.86E+13 Coefficients Standard error t-stat P-value Lower 95% Upper 95% Intercept 4501.670443 6116.644 0.7360163 0.47863-916.187 1819.56 µg/ml 3639.79988 130.14 78.505808 8.76E-1 35949.86941 3659.73
No. 4 (009) Page: 31 of 45 Date: June 009 4000000 3500000 3000000 µg/ml Line Fit Plot There is a highly significant correlation between area and concentration. R = 0.99979. The slope coefficient = 3640 and the intercept on the Y-axis = 450. Area 500000 000000 1500000 1000000 Area Predicted Area Line Fit A confidence interval has been calculated for the intercept, and this includes 0.0, which means the standard curve passes through 0.0. 500000 0 0 50 100 150 µg/ml The residual (the difference between the observed and predicted Y-value) has been plotted against the concentration. RESIDUAL-OUTPUT Observation Predicted area Residuals Res-square 1 95101.17015-3078.17 9475131.46 95101.17015-309.17 1098773 3 185700.6699 1547.3301 39430.58 4 185700.6699 45.33015 180905.733 5 366899.6693-985.669 96543776.5 6 366899.6693-11150.67 1433745 7 910496.6675 4830.335 33311.1 8 910496.6675 7394.335 54676153.1 9 1816491.665-8764.665 76819344.8 10 1816491.665 36697.335 134669449 11 368481.659-3900.66 57141485 1 368481.659 9034.3413 8161933. Sum 397613089 Residuals 40000 30000 0000 10000-30000 µg/ml Residual Plot 0-10000 0 50 100 150-0000 µg/m l s 1 = 39761309 The figure shows that the points are distributed randomly around the X-axis, which indicates that the curve really is linear. If the points curve either upwards or downwards, this indicates that the adjustment may in fact be a second degree polynomial.
No. 4 (009) Page: 3 of 45 Date: June 009 Linearity can also be tested according to Tiley: 1 1 s = ( ˆ) 1 y = 397613089 = 39761309 n y 10 1 s = ( y y) where n repeated determinations have been carried out at the same n 1 concentration level, preferably near the middle of the standard curve, or with the same number of repetitions for all points (here repetitions), it is possible to summarise over all the points of the standard curve: 1 1 s ( ) = a b = 3161113435 = 6346119.6 n 6 where a-b is the difference between the two repetitions, and n is the number of points on the curve, i.e. 6. s1 39761309 F = = = 0.91 with 10 and 6 degrees of freedom. F critical (p=0.05) = 4.06 s 6346119.6 Thus, the correlation is linear. 6.3. PRECISION (as described in 3.4) The standard deviation of the method s repeatability (s r ) and the standard deviation of the reproducibility (s R ) may be calculated in several ways. Here, we have chosen to demonstrate the so-called variance analysis model for calculating the internal reproducibility and repeatability of the method during validation. The content of vitamin C in a certain food, is analysed on 3-6 different days, here 6 days, with a suitable number of repetitions every day, here repetitions. Table 6.3 Results of replicates analysed over 6 days. Vitamin C in a vegetable mg/100 g Vitamin C Day 1 Day Day 3 Day 4 Day 5 Day 6 Repeat 1 59.00 54.50 5.1 53.43 54.70 55.4 Repeat 58.64 5.00 5.3 53.00 58.46 57.93 Choose Excel: Data analysis, Variance analysis/ Anova: Single factor. Mark Grouped by columns and mark Labels in first row. The variance analysis in Annex 3 (Excel spread sheet available at the www.nmkl.org under Download Excel spread sheet ) calculates s r (within groups) and s d (between groups).