Practical guide for the validation, quality control, and uncertainty assessment of an alternative oenological analysis method (Resolution 10/2005)

Transcription

1 Gude for the valdato qualty cotrol Practcal gude for the valdato, qualty cotrol, ad ucertaty assessmet of a alteratve oeologcal aalyss method (Resoluto 10/005) Cotets 1. PURPOSE PREAMBLE AND SCOPE GENERAL VOCABULARY GENERAL PRINCIPLES METHODOLOGY DEFINITION OF MEASUREMENT ERROR VALIDATING A METHOD METHODOLOGY SECTION ONE: SCOPE OF METHOD Defto of aalyzable matrces Detecto ad quatfcato lmt Normatve defto Referece documets Applcato Procedure Determato o blak Scope Basc protocol ad calculatos Approach by learty study Scope Basc protocol ad calculatos Graphc approach based o the backgroud ose of the recordg Scope Basc protocol ad calculato Checkg a predetermed quatfcato lmt Scope Basc protocol ad calculato Robustess Defto Determato SECTION TWO: SYSTEMATIC ERROR STUDY Learty study Normatve defto Referece documets Applcato... 4 OIV-MA-AS1-1 : R005 1

2 Gude for the valdato qualty cotrol ISO type approach Basc protocol Calculatos ad results Defg the regresso model Estmatg parameters Charts Test of the learty assumpto Deftos of errors lked to calbrato Fscher-Sedecor test ISO 8466-type approach Basc protocol Calculatos ad results Defg the lear regresso model Defg the polyomal regresso model Comparg resdual stadard devatos Specfcty Normatve defto Applcato Procedures Stadard addto test Scope Basc protocol Calculatos ad results Study of the regresso le r = a + b.v Aalyss of the results Overlap le graphcs Study of the fluece of other compouds o the measuremet result Scope Basc protocol ad calculatos Iterpretato Study of method accuracy Presetato of the step Defto Geeral prcples Referece documets Comparso of the alteratve method wth the OIV referece method Scope Accuracy of the alteratve method compared wth the referece method Defto Scope Basc protocol ad calculatos Iterpretato Comparso by terlaboratory tests Scope Basc protocol ad calculatos Iterpretato Comparso wth referece materals Scope Basc protocol ad calculatos Iterpretato SECTION THREE: RANDOM ERROR STUDY Geeral prcple... 5 OIV-MA-AS1-1 : R005

3 Gude for the valdato qualty cotrol 5.4. Referece documets Precso of the method Defto Scope Geeral theoretcal case Basc protocol ad calculatos Calculatos wth several test materals Calculatos wth 1 test materal Repeatablty Deftos Scope Basc protocol ad calculatos Geeral case Partcular case applcable to oly 1 repetto Comparso of repeatablty Determato of the repeatablty of each method Fscher-Sedecor test Itralaboratory reproducblty Defto Scope Basc protocol ad calculatos QUALITY CONTROL OF ANALYSIS METHODS (IQC) REFERENCE DOCUMENTS GENERAL PRINCIPLES REFERENCE MATERIALS CHECKING THE ANALYTICAL SERIES Defto Checkg accuracy usg referece materals Itraseres precso Iteral stadard CHECKING THE ANALYSIS SYSTEM Defto Shewhart chart Data acqusto Presetato of results ad defto of lmts Usg the Shewhart chart Iteral comparso of aalyss systems Eteral comparso of the aalyss system Aalyss cha of terlaboratory comparsos Comparso wth eteral referece materals Stadard ucertaty of referece materal Defg the valdty lmts of measurg referece materal ASSESSMENT OF MEASUREMENT UNCERTAINTY DEFINITION REFERENCE DOCUMENTS SCOPE METHODOLOGY OIV-MA-AS1-1 : R005 3

4 Gude for the valdato qualty cotrol Defto of the measurad, ad descrpto of the quattatve aalyss method Crtcal aalyss of the measuremet process Estmato calculatos of stadard ucertaty (tralaboratory approach) Prcple Calculatg the stadard devato of tralaboratory reproducblty Estmatg typcal sources of systematc errors ot take to accout uderµ reproducblty codtos Gaugg error (or calbrato error) Procedure Calculatos ad results Estmatg the stadard ucertaty assocated the gaugg le (or calbrato le) Bas error Methods adjusted wth oly oe certfed referece materal Methods adjusted wth several referece materals (gaugg rages etc) Matr effect Defto Sample effect Estmatg stadard ucertaty by terlaboratory tests Prcple Usg the stadard devato of terlaboratory ad tramethod reproducblty SR ter (method) Usg the stadard devato of terlaboratory ad termethod reproducblty SR ter Other compoets the ucertaty budget EXPRESSING EXPANDED UNCERTAINTY OIV-MA-AS1-1 : R005 4

5 Gude for the valdato qualty cotrol 1. Purpose The purpose of ths gude s to assst oeologcal laboratores carryg out seral aalyss as part of ther valdato, teral qualty cotrol ad ucertaty assessmet tatves cocerg the stadard methods they use.. Preamble ad scope Iteratoal stadard ISO 1705, defg the "Geeral Requremets for the Competece of Testg ad Calbrato Laboratores", states that the accredted laboratores must, whe mplemetg a alteratve aalytcal method, make sure of the qualty of the results obtaed. To do so, t dcates several steps. The frst step cossts defg the customers' requremets cocerg the parameter questo, order to determe, thereafter, whether the method used meets those requremets. The secod step cludes tal valdato for o-stadardzed, modfed or laboratory-developed methods. Oce the method s appled, the laboratores must use specto ad traceablty methods order to motor the qualty of the results obtaed. Fally, they must assess the ucertaty of the results obtaed. I order to meet these requremets, the laboratores have a sgfcat referece system at ther dsposal comprsg a large umber of teratoal gudes ad stadards. However, practce, the applcato of these tets s delcate sce, because they address every category of calbrato ad test laboratory, they rema very geeral ad presuppose, o behalf of the reader, -depth kowledge of the mathematcal rules applcable to statstcal data processg. Ths gude s based o ths teratoal referece system, takg to accout the specfc characterstcs of oeology laboratores routely carryg out aalyses o seres of must or we samples. Defg the scope of applcato ths way eabled a relevat choce of sutable tools to be made, order to reta oly those methods most sutable for that scope. Sce t s based o the teratoal referece system, ths gude s therefore strctly complat wth t. Readers, however, wshg to study certa pots of the gude greater detal ca do so by referrg to the teratoal stadards ad gudes, the refereces for whch are gve each chapter. The authors have chose to combe the varous tools meetg the requremets of the ISO 1705 stadard sce there s a obvous soluto of cotuty ther OIV-MA-AS1-1 : R005 5

6 Gude for the valdato qualty cotrol applcato, ad the data obtaed wth certa tools ca ofte be used wth the others. I addto, the mathematcal resources used are ofte smlar. The varous chapters clude applcato eamples, take from oeology laboratores usg these tools. It s mportat to pot out that that ths gude does ot preted to be ehaustve. It s oly desged to preset, as clear ad applcable a way as possble, the cotets of the requremets of the ISO 1705 stadard ad the basc resources that ca be mplemeted a route laboratory to meet them. Each laboratory remas perfectly free to supplemet these tools or to replace them by others that they cosder to be more effcet or more sutable. Fally, the reader s atteto should be draw to the fact that the tools preseted do ot costtute a ed themselves ad that ther use, as well as the terpretato of the results to whch they lead, must always be subject to crtcal aalyss. It s oly uder these codtos that ther relevace ca be guarateed, ad laboratores wll be able to use them as tools to mprove the qualty of the aalyses they carry out. 3. Geeral vocabulary The deftos dcated below used ths documet result from the ormatve refereces gve the bblography. Aalyte Object of the aalyss method Blak Test carred out the absece of a matr (reaget blak) or o a matr whch does ot cota the aalyte (matr blak). Bas Dfferece betwee the epected test results ad a accepted referece value. Ucertaty budget The lst of ucertaty sources ad ther assocated stadard ucertates, establshed order to assess the compoud stadard ucertaty assocated wth a measuremet result. OIV-MA-AS1-1 : R005 6

7 Gude for the valdato qualty cotrol Gaugg (of a measurg strumet) Materal postog of each referece mark (or certa prcpal referece marks oly) of a measurg strumet accordg to the correspodg value of the measurad. NOTE "gaugg" ad "calbrato" are ot be cofused Repeatablty codtos Codtos where depedet test results are obtaed wth the same method o detcal test tems the same laboratory by the same operator usg the same equpmet wth short tervals of tme. Reproducblty codtos (tralaboratory) Codtos where depedet test results are obtaed wth the same method o detcal test tems the same laboratory by the same or dfferet operator(s) usg dfferet gauges o dfferet days. Epermetal stadard devato For a seres of measuremets of the same measurad, the quatty s characterzg the dsperso of the results ad gve by the formula: s ( 1 ) 1 beg the result of the measuremet th ad the arthmetc mea of the results cosdered. Repeatablty stadard devato Stadard devato of may repettos obtaed a sgle laboratory by the same operator o the same strumet,.e. uder repeatable codtos. Iteral reproducblty stadard devato (or total tralaboratory varablty) Stadard devato of repettos obtaed a sgle laboratory wth the same method, usg several operators or strumets ad, partcular, by takg measuremets o dfferet dates,.e. uder reproducblty codtos. Radom error Result of a measuremet mus the mea that would result from a fte umber of measuremets of the same measurad carred out uder reproducblty codtos. OIV-MA-AS1-1 : R005 7

8 Gude for the valdato qualty cotrol Measuremet error Result of a measuremet mus a true value of the measurad. Systematc error Mea error that would result from a fte umber of measuremets of the same measurad carred out uder reproducblty codtos mus a true value of the measurad. NOTE Error s a hghly theoretcal cocept that t calls upo values that are ot accessble practce, partcular the true values of measurads. O prcple, the error s ukow. Mathematcal epectato For a seres of measuremets of the same measurad, f teds towards the fte, the mea teds towards the epectato E(). lm 1 E( ) Calbrato Seres of operatos establshg uder specfed codtos the relato betwee the values of the quatty dcated by a measurg strumet or system, or the values represeted by a materalzed measuremet or a referece materal, ad the correspodg values of the quatty measured by stadards. Itralaboratory evaluato of a aalyss method Acto whch cossts submttg a aalyss method to a tralaboratory statstcal study, based o a stadardzed ad/or recogzed protocol, demostratg that wth ts scope, the aalyss method meets pre-establshed performace crtera. Wth the framework of ths documet, the evaluato of a method s based o a tralaboratory study, whch cludes the comparso wth a referece method. Precso Closeess of agreemet betwee depedet test results obtaed uder prescrbed codtos NOTE 1 Precso depeds oly o the dstrbuto of radom errors ad does ot have ay relatoshp wth the true or specfed value. NOTE The measuremet of precso s epressed o the bass of the stadard devato of the test results. NOTE 3 The epresso "depedet test results" refers to results obtaed such that they are ot flueced by a prevous result o the same or a smlar test OIV-MA-AS1-1 : R005 8

9 Gude for the valdato qualty cotrol materal. Quattatve measuremets of precso are crtcally depedet upo the prescrbed codtos. Repeatablty ad reproducblty codtos are partcular sets of etreme codtos. Quatty (measurable) A attrbute of a pheomeo, body or substace that may be dstgushed qualtatvely ad determed quattatvely. Ucertaty of measuremet A parameter assocated wth the result of a measuremet, whch characterzes the dsperso of the values that could reasoably be attrbuted to the measurad. Stadard ucertaty (u()) Ucertaty of the result of a measuremet epressed the form of a stadard devato. Accuracy Closeess of agreemet betwee the mea value obtaed startg from a broad seres of test results ad a accepted referece value. NOTE The measuremet of accuracy s geerally epressed terms of bas. Detecto lmt Lowest amout of a aalyte to be eamed a test materal that ca be detected ad regarded as dfferet from the blak value (wth a gve probablty), but ot ecessarly quatfed. I fact, two rsks must be take to accout: - the rsk of cosderg the substace s preset test materal whe ts quatty s ull; - the rsk of cosderg a substace s abset from a substace whe ts quatty s ot ull. Quatfcato lmt Lowest amout of a aalyte to be eamed a test materal that ca be quattatvely determed uder the epermetal codtos descrbed the method wth a defed varablty (gve coeffcet of varato). Learty The ablty of a method of aalyss, wth a certa rage, to provde a strumetal respose or results proportoal to the qualty of aalyte to be determed the laboratory sample. Ths proportoalty s epressed by a a pror defed mathematcal epresso. OIV-MA-AS1-1 : R005 9

10 Gude for the valdato qualty cotrol The learty lmts are the epermetal lmts of cocetratos betwee whch a lear calbrato model ca be appled wth a kow cofdece level (geerally take to be equal to 1%). Test materal Materal or substace to whch a measurg ca be appled wth the aalyss method uder cosderato. Referece materal Materal or substace oe or more of whose property values are suffcetly homogeeous ad well establshed to be used for the calbrato of a apparatus, the assessmet of a measuremet method, or for assgg values to materals. Certfed referece materal Referece materal, accompaed by a certfcate, oe or more whose property values are certfed by a procedure whch establshes ts traceablty to a accurate realzato of the ut whch the property values are epressed, ad for whch each certfed value s accompaed by a ucertaty at a stated level of cofdece. Matr All the costtuets of the test materal other tha the aalyte. Aalyss method Wrtte procedure descrbg all the meas ad procedures requred to carry out the aalyss of the aalyte,.e.: scope, prcple ad/or reactos, deftos, reagets, apparatus, procedures, epresso of results, precso, test report. WARNING The epressos "ttrato method" ad "determato method" are sometmes used as syoyms for the epresso "aalyss method". These two epressos should ot be used ths way. Quattatve aalyss method Aalyss method makg t possble to measure the aalyte quatty preset the laboratory test materal. Referece aalyss method (Type I or Type II methods) Method, whch gves the accepted referece value for the quatty of the aalyte to be measured. No-classfed alteratve method of aalyss A route aalyss method used by the laboratory ad ot cosdered to be a referece method. OIV-MA-AS1-1 : R005 10

11 Gude for the valdato qualty cotrol NOTE A alteratve method of aalyss ca cosst a smplfed verso of the referece method. Measuremet Set of operatos havg the object of determg a value of a quatty. NOTE The operatos ca be carred out automatcally. Measurad Partcular quatty subject to measuremet. Mea For a seres of measuremets of the same measurad, mea value, gve by the formula: beg the result of the th measuremet. 1 Result of a measuremet Value assged to a measurad, obtaed by measuremet Sestvty Rato betwee the varato of the formato value of the aalyss method ad the varato of the aalyte quatty. The varato of the aalyte quatty s geerally obtaed by preparg varous stadard solutos, or by addg the aalyte to a matr. NOTE 1 Defg, by eteso, the sestvty of a method as ts capacty to detect small quattes should be avoded. NOTE A method s sad to be sestve" f a low varato of the quatty or aalyte quatty curs a sgfcat varato the formato value. Measuremet sgal Quatty represetg the measurad ad s fuctoally lked to t. Specfcty Property of a aalyss method to respod eclusvely to the determato of the quatty of the aalyte cosdered, wth the guaratee that the measured sgal comes oly from the aalyte. OIV-MA-AS1-1 : R005 11

12 Gude for the valdato qualty cotrol Tolerace Devato from the referece value, as defed by the laboratory for a gve level, wth whch a measured value of a referece materal ca be accepted. Value of a quatty Magtude of a partcular quatty geerally epressed as a ut of measuremet multpled by a umber. True value of a quatty Value compatble wth the defto of a gve partcular quatty. NOTE 1 NOTE The value that would be obtaed f the measuremet was perfect Ay true value s by ature determate Accepted referece value A value that serves as a agreed-upo referece for comparso ad whch s derved as: a) a theoretcal or establshed value, based o scetfc prcples; b) a assged or certfed value, based o epermetal work of some atoal or teratoal orgazato; c) a cosesus or certfed value, based o collaboratve epermetal work uder the auspces of a scetfc or egeerg group; Wth the partcular framework of ths documet, the accepted referece value (or covetoally true value) of the test materal s gve by the arthmetc mea of the values of measuremets repeated as per the referece method. Varace Square of the stadard devato. 4. Geeral prcples 4.1 Methodology Whe developg a ew alteratve method, the laboratory mplemets a protocol that cludes several steps. The frst step, appled oly oce at the tal stage, or o a regular bass, s the valdato of the method. Ths step s followed by permaet qualty cotrol. All the data collected durg these two steps make t possble to assess the qualty of the method. The data collected durg these two OIV-MA-AS1-1 : R005 1

13 Gude for the valdato qualty cotrol steps are used to evaluate the measuremet ucertaty. The latter, whch s regularly assessed, s a dcator of the qualty of the results obtaed by the method uder cosderato. Developmet or adopto of a method Step of tal valdato Settg-up ad mplemetato of the qualty cotrol system Ucertaty assessmet All these steps are ter-coected ad costtute a global approach that ca be used to assess ad cotrol measuremet errors. 4. Defto of measuremet error Ay measuremet carred out usg the method uder study gves a result whch s evtably assocated wth a measuremet error, defed as beg the dfferece betwee the result obtaed ad the true value of the measurad. I practce, the true value of the measurad s accessble ad a value covetoally accepted as such s used stead. The measuremet error cludes two compoets: Measuremet error True value = Aalyss result + Systematc error + Radom error OIV-MA-AS1-1 : R005 13

14 Gude for the valdato qualty cotrol I practce, the systematc error results a bas relato to the true value, the radom error beg all the errors assocated wth the applcato of the method. These errors ca be graphcally represeted the followg way: Gauss dstrbuto of the results Precso Error Systematc error Radom error True value Mea value of a fte umber of results Result of a aalyss The valdato ad qualty cotrol tools are used to evaluate the systematc errors ad the radom errors, ad to motor ther chages over tme. 5. Valdatg a method 5.1 Methodology Implemetg the valdato comprses 3 steps, each wth objectves. To meet these objectves, the laboratory has valdato tools. Sometmes there are may tools for a gve objectve, ad are sutable for varous stuatos. It s up to the laboratory to correctly choose the most sutable tools for the method to be valdated. OIV-MA-AS1-1 : R005 14

15 Gude for the valdato qualty cotrol Steps Objectves Tools for valdato Scope of applcato Systematc error or bas Radom error - To defe the aalyzable matrces - To defe the aalyzable rage Detecto ad quatfcato lmt Robustess study - Lear respose the scale of Learty study aalyzable values - Specfcty of the method Specfcty study - Accuracy of the method Comparso wth a referece method Comparso wth referece materals Iterlaboratory comparso - Precso of the method Repeatablty study Itralaboratory reproducblty study 5. Secto oe: Scope of method 5..1 Defto of aalyzable matrces The matr comprses all costtuets the test materal other tha the aalyte. If these costtuets are lable to fluece the result of a measuremet, the laboratory should defe the matrces o whch the method s applcable. For eample, oeology, the determato of certa parameters ca be flueced by the varous possble matrces (wes, musts, sweet wes, etc.). I case of doubt about a matr effect, more -depth studes ca be carred out as part of the specfcty study. 5.. Detecto ad quatfcato lmt Ths step s of course ot applcable ad ot ecessary for those methods whose lower lmt does ot ted towards 0, such as alcoholc stregth by volume wes, total acdty wes, ph, etc. OIV-MA-AS1-1 : R005 15

16 Gude for the valdato qualty cotrol Normatve defto The detecto lmt s the lowest amout of aalyte that ca be detected but ot ecessarly quatfed as a eact value. The detecto lmt s a parameter of lmt tests. The quatfcato lmt s the lowest quatty of the compoud that ca be determed usg the method Referece documets - NF V Stadard, tralaboratory valdato procedure for a alteratve method relato to a referece method. - Iteratoal compedum of aalyss methods Ŕ OIV, Assessmet of the detecto ad quatfcato lmt of a aalyss method (Oeo resoluto 7/000) Applcato I practce, the quatfcato lmt s geerally more relevat tha the detecto lmt, the latter beg by coveto 1/3 of the frst. There are several approaches for assessg the detecto ad quatfcato lmts: - Determato o blak - Approach by the learty study - Graphc approach These methods are sutable for varous stuatos, but every case they are mathematcal approaches gvg results of formatve value oly. It seems crucal, wheever possble, to troduce a check of the value obtaed, whether by oe of these approaches or estmated emprcally, usg the checkg protocol for a predetermed quatfcato lmt Procedure Determato o blak Scope OIV-MA-AS1-1 : R005 16

17 Gude for the valdato qualty cotrol Ths method ca be appled whe the blak aalyss gves results wth a o-zero stadard devato. The operator wll judge the advsablty of usg reaget blaks, or matr blaks. If the blak, for reasos related to ucotrolled sgal preprocessg, s sometmes ot measurable or does ot offer a recordable varato (stadard devato of 0), the operato ca be carred out o a very low cocetrato aalyte, close to the blak Basc protocol ad calculatos Carry out the aalyss of test materals assmlated to blaks, beg equal to or hgher tha Calculate the mea of the results obtaed: blak 1 - Calculate the stadard devato of the results obtaed: Sblak ( blak ) From these results the detecto lmt s covetoally defed by the formula: L d blak ( 3. Sblak) - From these results the quatfcato lmt s covetoally defed by the formula: L q blak ( 10. Sblak) Eample: The table below gves some of the results obtaed whe assessg the detecto lmt for the usual determato of free sulfur dode. OIV-MA-AS1-1 : R005 17

18 Gude for the valdato qualty cotrol Test materal # X ( mg/l) The calculated values are as follows: q = 1 M blak = S blak = 0.58 mg/l DL = 1.96 mg/l QL = 5.65 mg/l Approach by learty study Scope Ths method ca be appled all cases, ad s requred whe the aalyss method does ot volve backgroud ose. It uses the data calculated durg the learty study. NOTE Ths statstcal approach may be based ad gve pessmstc results whe learty s calculated o a very wde rage of values for referece materals, ad whose measuremet results clude varable stadard devatos. I such cases, a learty study lmted to a rage of low values, close to 0 ad wth a more homogeeous dstrbuto wll result a more relevat assessmet. OIV-MA-AS1-1 : R005 18

19 Gude for the valdato qualty cotrol Basc protocol ad calculatos Use the results obtaed durg the learty study whch made t possble to calculate the parameters of the calbrato fucto y = a+ b. The data to be recovered from the learty study are (see chapter learty study): - slope of the regresso le: ( M )( y M y) b 1 ( M ) 1 - resdual stadard devato: Sres p 1 j1 y yˆ (, j, j p ) - stadard devato at the tercept pot (to be calculated): 1 Sa Sres p p 1 M ( M ) The estmates of the detecto lmt DL ad the quatfcato lmt QL are calculated usg followg formulae: DL QL S 3 a b S 10 a b Estmated detecto lmt Estmated quatfcato lmt OIV-MA-AS1-1 : R005 19

20 Gude for the valdato qualty cotrol Eample: Estmatato of the detecto ad quatfcato lmts the determato of sorbc acd by capllary electrophoress, based o learty data acqured o a rage from 1 to 0 mg.l -1. X (ref) Y1 Y Y3 Y Number of referece materals = 8 Number of replcas p = 4 Straght le (y = a + b*) b = a = resdual stadard devato: S res = Stadard devato o the tercept pot S a = The estmated detecto lmt s DL = 0.48 mg.l -1 The estmated quatfcato lmt s QL = 1.6 mg.l -1 OIV-MA-AS1-1 : R005 0

21 Gude for the valdato qualty cotrol Graphc approach based o the backgroud ose of the recordg Scope Ths approach ca be appled to aalyss methods that provde a graphc recordg (chromatography, etc.) wth a backgroud ose. The lmts are estmated from a study of the backgroud ose Basc protocol ad calculato Record a certa umber of reaget blaks, usg 3 seres of 3 jectos separated by several days. Determe the followg values: h ma the greatest varato ampltude o the y-as of the sgal observed betwee two acqusto pots, ecludg drft, at a dstace equal to twety tmes the wdth at md-heght of the peak correspodg to the aalyte, cetered over the reteto tme of the compoud uder study. R, the quatty/sgal respose factor, epressed heght. The detecto lmt DL, ad the quatfcato lmt QL are calculated accordg to the followg formulae: DL = 3 h ma R QL = 10 h ma R Checkg a predetermed quatfcato lmt Ths approach ca be used to valdate a quatfcato value obtaed by statstcal or emprcal approach Scope Ths method ca be used to check that a gve quatfcato lmt s a pror acceptable. It s applcable whe the laboratory ca procure at least 10 test materals wth kow quattes of aalyte, at the level of the estmated quatfcato lmt. I the case of methods wth a specfc sgal, ot sestve to matr effects, the materals ca be sythetc solutos whose referece value s obtaed by formulato. OIV-MA-AS1-1 : R005 1

22 Gude for the valdato qualty cotrol I all other cases, wes (or musts) shall be used whose measurad value as obtaed by the referece method s equal to the lmt to be studed. Of course, ths case the quatfcato lmt of the referece method must be lower tha ths value Basc protocol ad calculato Aalyze depedet test materals whose accepted value s equal to the quatfcato lmt to be checked; must at least be equal to Calculate the mea of measuremets: LQ 1 - Calculate the stadard devato of measuremets: SLQ 1 ( LQ ) 1 wth results of the measuremet of the th test materal. The two followg codtos must be met: a) the measured mea quatty LQ must ot be dfferet from the predetermed quatfcato lmt QL: If QL Ql < 10 the quatfcato lmt QL s cosdered to be vald. S QL NOTE 10 s a purely covetoal value relatg to the QL crtero. b) the quatfcato lmt must be other tha 0: If 5 s QL < QL the the quatfcato lmt s other tha 0. A value of 5 correspods to a appromate value for the spread of the stadard devato, takg to accout rsk ad rsk to esure that the QL s other tha 0. OIV-MA-AS1-1 : R005

23 Gude for the valdato qualty cotrol Ths s equvalet to checkg that the coeffcet of varato for QL s lower tha 0%. NOTE1 Remember that the detecto lmt s obtaed by dvdg the quatfcato lmt by 3. NOTE A check should be made to esure that the value of S LQ s ot too large (whch would produce a artfcally postve test), ad effectvely correspods to a reasoable stadard devato of the varablty of the results for the level uder cosderato. It s up to the laboratory to make ths crtcal evaluato of the value of S LQ. Eample: Checkg the quatfcato lmt of the determato of malc acd by the ezymatc method. Estmated quatfcato lmt: 0.1 g.l -1 We Values Mea: Stadard devato: Frst codto: LQQL The quatfcato S QL lmt of 0.1 s cosdered to be vald. Secod codto: 5. S LQ The quatfcato lmt s cosdered to be sgfcatly dfferet from 0. OIV-MA-AS1-1 : R005 3

24 Gude for the valdato qualty cotrol 5..3 Robustess Defto Robustess s the capacty of a method to gve close results the presece of slght chages the epermetal codtos lkely to occur durg the use of the procedure Determato If there s ay doubt about the fluece of the varato of operatoal parameters, the laboratory ca use the scetfc applcato of epermet schedules, eablg these crtcal operatg parameters to be tested wth the varato rage lkely to occur uder practcal codtos. I practce, these tests are dffcult to mplemet. 5.3 Secto two: systematc error study Learty study Normatve defto The learty of a method s ts ablty (wth a gve rage) to provde a formatve value or results proportoal to the amout of aalyte to be determed the test materal Referece documets - NF V stadard. Itralaboratory valdato procedure of a alteratve method relato to a referece method. - ISO Stadard, lear calbrato usg referece materals. - ISO Stadard, Water qualty Ŕ Calbrato ad evaluato of aalytcal methods ad estmato of performace characterstcs Applcato The learty study ca be used to defe ad valdate a lear dyamc rage. Ths study s possble whe the laboratory has stable referece materals whose accepted values have bee acqured wth certaty ( theory these values should have a ucertaty equal to 0). These could therefore be teral referece materals ttrated wth calbrated materal, wes or musts whose value s gve by OIV-MA-AS1-1 : R005 4

25 Gude for the valdato qualty cotrol the mea of at least 3 repettos of the referece method, eteral referece materals or certfed eteral referece materals. I the last case, ad oly ths case, ths study also eables the traceablty of the method. The epermet schedule used here could the be cosdered as a calbrato. I all cases, t s advsable to esure that the matr of the referece materal s compatble wth the method. Lastly, calculatos must be made wth the fal result of the measuremet ad ot wth the value of the sgal. Two approaches are proposed here: - A ISO type of approach, the prcple of whch cossts comparg the resdual error wth the epermetal error usg a Fscher's test. Ths approach s vald above all for relatvely arrow rages ( whch the measurad does ot vary by more tha a factor 10). I addto, uder epermetal codtos geeratg a low reproducblty error, the test becomes ecessvely severe. O the other had, the case of poor epermetal codtos, the test wll easly be postve ad wll also lose ts relevace. Ths approach requres good homogeety of the umber of measuremets over the etre rage studed. - A ISO 8466 type of approach, the prcple of whch cossts comparg the resdual error caused by the lear regresso wth the resdual error produced by a polyomal regresso (of order for eample) appled to the same data. If the polyomal model gves a sgfcatly lower resdual error, a cocluso of olearty could be draw. Ths approach s approprate partcular whe there s a rsk of hgh epermetal dsperso at oe ed of the rage. It s therefore aturally well-suted to aalyss methods for traces. There s o eed to work wth a homogeeous umber of measuremets over the whole rage, ad t s eve recommeded to crease the umber of measuremets at the borders of the rage ISO type approach Basc protocol It s advsable to use a umber of referece materals. The umber must be hgher tha 3, but there s o eed, however, to eceed 10. The referece materals OIV-MA-AS1-1 : R005 5

26 Gude for the valdato qualty cotrol should be measured p tmes, uder reproducblty codtos, p shall be hgher tha 3, a umber of 5 beg geerally recommeded. The accepted values for the referece materals are to be regularly dstrbuted over the studed rage of values. The umber of measuremets must be detcal for all the referece materals. NOTE It s essetal that the reproducblty codtos use a mamum of potetal sources of varablty, wth the rsk that the test shows o-learty a ecessve way. The results are reported a table preseted as follows: Referece materals Accepted referece value materal Measured values... Replca... Replca j p Replca y y 1j... y 1p y 1... y j... y p y 1... y j... y p Calculatos ad results Defg the regresso model The model to be calculated ad tested s as follows: y a b. j j where y j b a a. s the j th replca of the th referece materal. s the accepted value of the th referece materal. s the slope of the regresso le. s the tercept pot of the regresso le. b represets the epectato of the measuremet value of the th referece materal. j s the dfferece betwee y j ad the epectato of the measuremet value of the th referece materal. OIV-MA-AS1-1 : R005 6

27 Gude for the valdato qualty cotrol Estmatg parameters The parameters of the regresso le are obtaed usg the followg formulae: - mea of p measuremets of the th referece materal 1 y p p j1 y j 1 - mea of all the accepted values of referece materals M 1 - mea of all the measuremets My 1 y 1 - estmated slope b b ( M 1 ( M ) 1 )( y M ) y - estmated tercept pot a a M y b M - regresso value assocated wth the th referece materal ŷ yˆ a b y yˆ - resdual e j j j e Charts The results ca be preseted ad aalyzed graphc form. Two types of charts are used. OIV-MA-AS1-1 : R005 7

28 Gude for the valdato qualty cotrol - The frst type of graph s the represetato of the values measured agast the accepted values of referece materals. The calculated overlap le s also plotted. Overlap le 10,00 8,00 s e lu v a6,00 d u re a s e M4,00 y= y^(régresso) replca que 1 1 replca que replca repl que3 3 replca que4 4,00 0,00 0,00,00 4,00 6,00 8,00 10,00 Accepted values of the referece materals - The secod graph s the represetato of the resdual values agast the estmated values of the referece materals ( ŷ ) dcated by the overlap le. The graph s a good dcator of the devato relato to the learty assumpto: the lear dyamc rage s vald f the resdual values are farly dstrbuted betwee the postve ad egatve values. OIV-MA-AS1-1 : R005 8

29 Resdual values COMPENDIUM OF INTERNATIONAL ANALYSIS OF METHODS - OIV Gude for the valdato qualty cotrol Resdual values relato to adjusted values: case of a o-lear method 0,6 0,5 0,4 0,3 0, 0, ,1 rep rep rep rep -0, -0,3-0,4 Adjusted values y^ I case of doubt about the learty of the regresso, a Fscher-Sedecor test ca be carred out order to test the assumpto: "the lear dyamc rage s ot vald", addto to the graphc aalyss Test of the learty assumpto Several error values lked to calbrato should be defed frst of all: these ca be estmated usg the data collected durg the epermet. A statstcal test s the performed o the bass of these results, makg t possble to test the assumpto of o-valdty of the lear dyamc rage: ths s the Fscher- Sedecor test Deftos of errors lked to calbrato These errors are gve as a stadard devato, resultg from the square root of the rato betwee a sum of squares ad a degree of freedom. Resdual error The resdual error correspods to the error betwee the measured values ad the value gve by the regresso le. The sum of the squares of the resdual error s as follows: p Qres ( y j yˆ ) 1 j1 OIV-MA-AS1-1 : R005 9

30 Gude for the valdato qualty cotrol The umber of degrees of freedom s p-. The resdual stadard devato s the estmated by the formula: Sres p ( y j yˆ ) 1 j1 p Epermetal error The epermetal error correspods to the reproducblty stadard devato of the epermetato. The sum of the squares of the epermetal error s as follows: The umber of degrees of freedom s p-. p Qep ( y j y ) 1 j1 The epermetal stadard devato (reproducblty) s the estmated by the formula: Sep p ( y j y ) 1 j1 p NOTE Ths quatty s sometmes also oted S R. Adjustmet error The value of the adjustmet error s the epermetal error mus the resdual error. The sum of the squares of the adjustmet error s: Q Q or def res p Qdef 1 j 1 ( y The umber of degrees of freedom s - j Q ep yˆ ) p ( y 1 j 1 j y ) OIV-MA-AS1-1 : R005 30

31 Gude for the valdato qualty cotrol The stadard devato of the adjustmet error s estmated by the formula: or Sdef Sdef Qres Qep p ( y 1 j1 j yˆ ) p ( y 1 j1 j y ) Fscher-Sedecor test The rato S ep freedom -, p-. S def F obeys the Fscher-Sedecor law wth the degrees of obs The calculated epermetal value F obs s compared wth the lmt value: F 1-α (-,p-), etracted from the Sedecor law table. The value for α used practce s geerally 5%. If F obs F 1-α the assumpto of the o-valdty of the lear dyamc rage s accepted (wth a rsk of α error of 5%). If F obs < F 1-α the assumpto of the o-valdty of the lear dyamc rage s rejected Eample: Learty study for the determato of tartarc acd by capllary electrophoress. 9 referece materals are used. These are sythetc solutos of tartarc acd, ttrated by meas of a scale traceable to stadard masses. Ref. materal T (ref) Y1 Y Y3 Y OIV-MA-AS1-1 : R005 31

32 Gude for the valdato qualty cotrol Regresso le Le ( y = a + b*) b = a = Errors related to calbrato Resdual stadard devato S res = Stadard devato of epermetal reproducblty S ep = Stadard devato of the adjustmet error S def = Iterpretato, Fscher-Sedecor test F obs = 0.53 < F 1-α =.37 The assumpto of the o-valdty of the lear dyamc rage s rejected ISO 8466-type approach Basc protocol It s advsable to use a umber of referece materals. The umber must be hgher tha 3, but there s o eed, however, to eceed 10. The referece materals should be measured several tmes, uder reproducblty codtos. The umber of measuremets may be small at the ceter of the rage studed (mmum = ) ad must be greater at both eds of the rage, for whch a mmum umber of 4 s geerally recommeded. The accepted values of referece materals must be regularly dstrbuted over the studed rage of values. NOTE It s vtal that the reproducblty codtos use the mamum umber of potetal sources of varablty. The results are reported a table preseted as follows: Referece materals Accepted value of the referece materal Measured values Replca j Replca 1 Replca... Replca p 1 1 y 11 y 1 y 1j... y 1p y 1 y N y 1... y j... y p OIV-MA-AS1-1 : R005 3

33 Gude for the valdato qualty cotrol OIV-MA-AS1-1 : R Calculatos ad results Defg the lear regresso model Calculate the lear regresso model usg the calculatos detaled above. The resdual error of the stadard devato for the lear model S res ca the be calculated usg the formula dcated Defg the polyomal regresso model The calculato of the polyomal model of order s gve below The am s to determe the parameters of the polyomal regresso model of order applcable to the data of the epermet schedule. c b a y The purpose s to determe the parameters a, b ad c. Ths determato ca geerally be computerzed usg spreadsheets ad statstcs software. The estmato formulae for these parameters are as follows: y y y y y N N N N a y y y y y N N N N b

34 Gude for the valdato qualty cotrol OIV-MA-AS1-1 : R y y y y y N N c Oce the model has bee establshed, the followg values are to be calculated: - regresso value assocated wth the th referece materal y ˆ c b a y ˆ - resdual e j j j y y e ˆ Resdual stadard devato of the polyomal model 1 1 ) ˆ ( p p j res y j y S Comparg resdual stadard devatos Calculato of S S DS res res N N 3) ( ) ( The S DS res PG The value PG s compared wth the lmt value F 1-α gve by the Fscher-Sedecor table for a cofdece level 1-α ad a degree of freedom 1 ad (N-3). NOTE I geeral the α rsk used s 5%. I some cases the test may be optmstc ad a rsk of 10% wll prove more realstc. If PG F 1-α : the olear calbrato fucto does ot result a mproved adjustmet; for eample, the calbrato fucto s lear. If PG > F 1-α : the work scope must be as arrow as possble to obta a lear calbrato fucto: otherwse, the formato values from the aalyzed samples must be evaluated usg a olear calbrato fucto.

35 Gude for the valdato qualty cotrol Eample: Theoretcal case. T (ref) Y1 Y Y3 Y Lear model ad polyomal model, method: theoretcal case Mea of me ase 50 ure d val ues y= y^(l reg) y'^(poly reg) y (mea) Accepted values for referece materals Lear regresso y = Ŕ S res = Polyomal regresso y = ² Ŕ S'res = Fscher's test PG = > F(5%) = PG>F the lear calbrato fucto caot be retaed OIV-MA-AS1-1 : R005 35

36 Gude for the valdato qualty cotrol 5.3. Specfcty Normatve defto The specfcty of a method s ts ablty to measure oly the compoud beg searched for Applcato I case of doubt about the specfcty of the tested method, the laboratory ca use epermet schedules desged to check ts specfcty. Two types of complemetary epermets are proposed here that ca be used a large umber of cases ecoutered the feld of oeology. - The frst test s the stadard addto test. It ca be used to check that the method measures all the aalyte. - The secod test ca be used to check the fluece of other compouds o the result of the measuremet Procedures Stadard addto test Scope Ths test ca be used to check that the method measures all the aalyte. The epermet schedule s based o stadard addtos of the compoud beg searched for. It ca oly be appled to methods that are ot sestve to matr effects Basc protocol Ths cossts fdg a sgfcat degree of added quattes o test materals aalyzed before ad after the addtos. Carry out varable stadard addtos o test materals. The tal cocetrato aalyte of test materals, ad the stadard addtos are selected order to cover the scope of the method. These test materals must cosst of the types of matrces called for route aalyss. It s advsed to use at least 10 test materals. The results are reported a table preseted as follows: OIV-MA-AS1-1 : R005 36

37 Gude for the valdato qualty cotrol Test materal Quatty before addto () Quatty added (v) Quatty after addto (w) Quatty foud (r) 1 1 v 1 w 1 r 1 = w 1 Ŕ v w r = w X V w r p = w NOTE 1 NOTE NOTE 3 A addto s made wth a pure stadard soluto. It s advsed to perform a addto of the same order as the quatty of the test materal o whch t s carred out. Ths s why the most cocetrated test materals must be dluted to rema wth the scope of the method. It s advsed to prepare the addtos usg depedet stadard solutos, order to avod ay systematc error. The qualty of values ad w ca be mproved by usg several repettos Calculatos ad results The prcple of the measuremet of specfcty cossts studyg the regresso le r = a + b.v ad checkg that slope b s equvalet to 1 ad that tercept pot a s equvalet to Study of the regresso le r = a + b.v The parameters of the regresso le are obtaed usg the followg formulae: - mea of the added quattes v - mea of the quattes foud r v 1 v r r 1 OIV-MA-AS1-1 : R005 37

38 Gude for the valdato qualty cotrol OIV-MA-AS1-1 : R estmated slope b v v r r v v b 1 1 ) ( ) )( ( - estmated tercept pot a v b r a. - regresso value assocated wth the th referece materal ŷ v b a r ˆ - resdual stadard devato ˆ 1 r r S res - stadard devato o the slope res b v v S S 1 ) ( 1 - stadard devato o the tercept pot res a v v v S S 1 ) ( Aalyss of the results The purpose s to coclude o the absece of ay terferece ad o a acceptable specfcty. Ths s true f the overlap le r = a + bv s equvalet to the le y =.

39 Gude for the valdato qualty cotrol To do so, two tests are carred out: - Test of the assumpto that slope b of the overlap le s equal to 1. - Test of the assumpto that tercept pot a s equal to 0. These assumptos are tested usg a Studet test, geerally assocated wth a rsk of error of 1%. A rsk of 5% ca prove more realstc some cases. Let T crtcal, blateral [dof; 1%] be a Studet blateral varable assocated wth a rsk of error of 1% for a umber of degrees of freedom (dof). Step 1: calculatos Calculato of the comparso crtero o the slope at 1 b1 Tobs Sb Calculato of the comparso crtero o the tercept pot at 0 a T' obs Sa Calculato of the Studet crtcal value: T crtcal, blateral [ p-; 1%] Step : terpretato If T obs s lower tha T crtcal, the the slope of the regresso le s equvalet to 1 If T obs s lower tha T crtcal, the the tercept pot of the regresso le s equvalet to 0. If both codtos are true, the the overlap le s equvalet = y =, ad the method s deemed to be specfc. NOTE 1 Based o these results, a mea overlap rate ca be calculated to quatfy the specfcty. I o case should t be used to "correct" the results. Ths s because f a sgfcat bas s detected, the alteratve method caot be valdated relato to a effcecy rate of 100%. NOTE Sce the prcple of the test cossts calculatg a straght le, at least three levels of addto have to be take, ad ther value must be correctly chose order to obta a optmum dstrbuto of the pots. OIV-MA-AS1-1 : R005 39

40 Y: Cotet foud Y: Cotet foud COMPENDIUM OF INTERNATIONAL ANALYSIS OF METHODS - OIV Gude for the valdato qualty cotrol Overlap le graphcs Eample of specfcty The specfcty s accepted The specfcty s ot accepted Y = X Estmated relato betwee the added cotet ad the cotet foud X : Added cotet foud Estmated relato betwee the added cotet ad the cotet foud 4 Y = X X : Cotet foud Study of the fluece of other compouds o the measuremet result Scope If the laboratory suspects the teracto of compouds other tha the aalyte, a epermet schedule ca be set up to test the fluece of varous compouds. The epermet schedule proposed here eables a search for the fluece of compouds defed a pror: thaks to ts kowledge of the aalytcal process ad ts kow-how, the laboratory should be able to defe a certa umber of compouds lable to be preset the we ad to fluece the aalytcal result Basc protocol ad calculatos Aalyze wes duplcate, before ad after the addto of the compoud suspected of havg a fluece o the aalytcal result; must at least be equal to 10. The mea values M of the measuremets ad ' made before the addto shall be calculated frst, the the mea values My of the measuremets y ad y' made after the addto, ad fally the dfferece d betwee the values M ad My. The results of the epermet ca be reported as dcated the followg table: OIV-MA-AS1-1 : R005 40

41 Gude for the valdato qualty cotrol : Before addto y: After addto Meas Dfferece Samples Rep1 Rep Rep1 Rep y d y 1 y 1 M 1 My 1 d 1 = M 1 - My y y M My d = M- My y y M My d = M - My The mea of the results before addto M 1 M M 1 The mea of the results after addto M y M 1 y My 1 Calculate the mea of the dffereces M d M d d My M 1 Calculate the stadard devato of the dffereces S d S d ( d M d ) 1 1 Calculate the Z-score Z score M S d d Iterpretato OIV-MA-AS1-1 : R005 41

42 Gude for the valdato qualty cotrol If the Z score s, the added compoud ca be cosdered to have a eglgble fluece o the result of aalyss wth a rsk of 5%. If the Z score s, the added compoud ca be cosdered to fluece the result of aalyss wth a rsk of 5%. NOTE Iterpretg the Z score s possble gve the assumpto that the varatos obey a ormal law wth a 95% cofdece rate. Eample: Study of the teracto of compouds lable to be preset the samples, o the determato of fructose glucose wes by Fourer trasform frared spectrophotometry (FTIR). v Before addto + 50 mg.l - 1 potassum sorbate + 1 g. L -1 salcylc acd rep1 rep rep1 rep rep1 rep Dffereces sorbate dff salcylc dff Potassum sorbate Md = 0.0 Sd = Z score = 0.3 < Salcylc acd Md = Sd = 0.8 Z score =.57 > I cocluso, t ca be stated that potassum sorbate does ot fluece the determato of fructose glucose by the FTIR gaugg studed here. O the other had, salcylc acd has a fluece, ad care should be take to avod OIV-MA-AS1-1 : R005 4