Maurice Cox, Peter Harris National Physical Laboratory, UK Steve Ellison LGC Ltd, UK

Transcription

1 CCQM/11-18 Use of an excess-varance aroach for the estmaton of a key comarson reference value, assocated standard uncertanty and degrees of equvalence for CCQM key comarson data Maurce Cox, Peter Harrs Natonal Physcal Laboratory, UK Steve Ellson LGC Ltd, UK Executve summary Ths document llustrates the use of an excess-varance aroach for the statstcal analyss of CCQM key comarson data that makes allowance for unexlaned laboratory effects. In excess-varance aroaches, the varances (squared standard uncertantes) rovded by artcatng laboratores are augmented by an addtonal varance, common to all laboratores, whch s regarded as characterzng unexlaned laboratory effects. The reorted varances are combned wth an estmate of the excess varance to rovde weghts that are used to estmate the key comarson reference value (KCRV). The uncertanty assocated wth ths estmate takes account of both the laboratory uncertantes and the estmated excess varance. For mutually consstent data the aroach reduces to the classcal weghted mean. Degrees of equvalence are determned accordngly. The resent document demonstrates the general aroach by reference to a artcular mlementaton, the DerSmonan-Lard method, aled to some llustratve examles of CCQM key comarsons. Suortng software can be made avalable to carry out the calculatons should CCQM agree wth the content of ths document.

2 1 Introducton Ths document s concerned wth a CCQM key comarson n whch each laboratory ndeendently rovdes a measured value and an assocated standard uncertanty for one or more measurands. Techncal revew of the data by the relevant workng grou dentfes those laboratory data that are consdered arorate for use n estmatng the KCRV. For many studes, ths revew results n a set of values and uncertantes that nclude no serous outlyng values, but that may show a dserson of data that cannot be fully exlaned by the reorted uncertantes. Whle a varety of methods can be aled to estmate a KCRV and ts assocated standard uncertanty n these crcumstances, one class of estmators that aears romsng s a class referred to n ths document as excess-varance aroaches. Snce the stuaton descrbed excess dserson wth no evdence of serous outlyng values s one of the general scenaros dentfed n draft gudance document CCQM [], the resent document can be consdered a secalzed case wthn CCQM Document CCQM remans alcable as a background document to be used for those sets of key comarson data for whch t s judged narorate to aly the aroach here. In excess-varance aroaches, the varances (squared standard uncertantes) rovded by artcatng laboratores are augmented by an addtonal varance, assumed to be common to all laboratores, whch s regarded as characterzng unexlaned laboratory effects. The reorted varances are combned wth the estmated excess varance to rovde weghts used to estmate the KCRV. The standard uncertanty assocated wth the KCRV takes account of both the laboratory varances and the estmated excess varance. For mutually consstent data the aroach reduces to the classcal weghted mean. The resent document uses a artcular estmator n ths class, the DerSmonan-Lard estmator, to llustrate the use of such an estmator. The aroach draws on rncles gven n document CCQM [1]. Due account s taken of the GUM [7]. Though the aroach llustrated here s straghtforward, t s not ntended to relace crtcal evaluaton of the data and arorate use of measurement and statstcal exertse, nor should t be regarded as rescrtve. However, the general aroach s lkely to be alcable to data for many CCQM key comarsons. Such an aroach would be comatble wth the remt of the CCQM KCRV WG n attemtng to harmonze as much as ossble the calculaton of the KCRV and ts assocated uncertanty Secton gves the suggested aroach and Secton 3 rovdes suortng techncal nformaton. The strengths and weaknesses of the aroach based on the DerSmonan-Lard estmator comared to the use of some other estmators are consdered n Secton 4. Secton 5 makes concludng remarks. Annexes contan the ratonale for the choce of estmator and llustratve examles. Model aroach The suggested aroach s based on the use of weghted means wth an arorate choce of weghts. An nterlaboratory varance s estmated, whch s used to augment the above laboratory varances and hence adjust the weghts. Ths addtonal varance term s chosen so that the combnaton of reorted standard uncertantes and estmated addtonal varance s suffcent to account for the observed dserson of values. The arorate weghted mean s taken as the KCRV, the standard uncertanty assocated wth the KCRV s evaluated, and DoEs accordngly determned. The aroach s descrbed below as a sequence of stes n whch the measured values and assocated

3 standard uncertantes rovded by the artcatng laboratores are denoted by x and u, resectvely. Cted equatons and formulae are gven n Table 1. The stes nvolved are as follows: 1. For uroses of the calculaton (but not for reortng), relace the u by the orgnal u augmented n quadrature by standard uncertantes relatng to any well-quantfed nhomogenety or nstablty effects.. Carefully examne the artcants data, makng use of grahcal tools and any statstcal tests (ncludng, as arorate, tests for nconsstency and outlyng values), as an ad to dentfyng ossble anomales. The workng grou may exclude anomalous data from the calculaton of the KCRV on techncal grounds. (For a more detaled dscusson see CCQM [].) 3. For the remanng data (x, u ), = 1,,, use formulae (1) to (3) to determne a value λ for the nterlaboratory varance and the DerSmonan-Lard (DL) mean x DL. Evaluate the standard uncertanty u(x DL ) assocated wth x DL usng formula (4). 4. Take x DL and u(x DL ) as the KCRV and ts assocated standard uncertanty. 5. Form the DoEs for the laboratores, usng formula (5) or (6), as arorate. Note: The same general formulae aly to some other excess-varance estmators Table 1. Formulae used n suggested aroach Parameter, etc. Formula Ref 1 Graybll-Deal mean x GD w x, w 1/ u, 1,...,, W1 w. (1) W 1 1 w ( x xgd) 1 1 Interlaboratory varance max0,, W w. 1 / 1 () W W W 1 1 N 1 ~ ~ ( u ) DerSmonan-Lard mean x DL w x, w. 1 1 x DL ( u j ) ~ ~ Standard uncertanty u(x DL ) u( xdl) w ( x xdl) (1 w ). 1 DoEs (d, u(d )) j 1 For laboratory data used for KCRV calculaton, 1/ d x x DL, u d ) u u ( x ). ( DL For data not used for KCRV calculaton, d x x DL, u d ) u u ( x ). ( DL (3) (4) (5) (6) 3

4 3 Suortng techncal nformaton The techncal nformaton here ales generally to excess-varance estmators and n artcular to the ste-by-ste aroach n Secton. 1. In Ste 1, samles measured by the laboratores corresond to dfferent measurands. Heterogenety or nstablty causes dfferences between a common measurand and the measurands corresondng to the samles. The augmented uncertantes take nto account the need to relate the measured values to a common measurand.. Although deally nterlaboratory effects should have a scentfc exlanaton (heterogenety or nstablty of samles, for nstance), such an exlanaton s often not forthcomng. Another ossble cause of nconsstency may be under-stated uncertantes u for some or all comarson artcants. 3. For a mutually consstent data set, the nterlaboratory varance λ s taken as zero. In a case of nconsstency, as λ s ncreased, the nfluence of those laboratores that rovde the smallest uncertantes s reduced. For extremely nconsstent data sets, λ becomes very large comared wth the u n order to acheve consstency, and the DL mean wll aroach the arthmetc mean. There s a smooth transton from the classcal weghted mean to the arthmetc mean as the data nconsstency ncreases. 4. The standard uncertanty u(x DL ) assocated wth x DL, determned n Ste 4, s comatble wth the augmented standard uncertantes (u + λ) 1/ assocated wth the measured values x. These standard uncertantes can be nterreted as osteror standard uncertantes assocated wth the x. These osteror standard uncertantes are necessary to overcome nconsstency n the data set (x, u ), = 1,, [14]. 5. The DoE for the th laboratory (Stes 5) conssts of a value comonent d and an uncertanty comonent U(d ) (at the 95 % level of confdence). The uncertanty comonent s exressed as U(d ) = u(d ), where u(d ) s the standard uncertanty assocated wth d, under a normalty assumton. 6. Formulae (5) and (6), used for calculatng DoE uncertantes, are comatble wth the osteror uncertantes assocated wth the x. In artcular, u(d ) obtaned from formula (5) does not reflect only the measurement uncertanty stated by laboratory. Formula (5) s chosen to be fully consstent wth the formal defnton of the DoE gven n the Techncal Annexe to the CIPM MRA that s, t s the standard uncertanty assocated wth the dfference between the reorted measured value and the KCRV after takng account of all terms n the statstcal model undernnng the KCRV. 4 Advantages and dsadvantages of excess-varance estmators Detaled ratonale for choosng the DerSmonan-Lard estmator for ths llustraton s gven n Annex A. Other excess-varance estmators have, however, been roosed for the same general roblem, n artcular the Mandel-Paule estmate and varatons on maxmum lkelhood estmaton. Some of the ractcal advantages and dsadvantages are lsted below. 1. The DerSmonan-Lard aroach has the advantage that t can be mlemented n a sngle smle calculaton and does not requre teratve soluton. The rncal dsadvantage s that t s a oorer aroxmaton than those rovded by more sohstcated teratve methods. It does not take account of fnte degrees of freedom n the ndvdual reorted uncertantes, and because t s equvalent to the Graybll-Deal estmator (the classcal weghted mean) [5] when the dserson s fully accounted for 4

5 by the reorted uncertantes, may substantally understate the uncertanty assocated wth the KCRV when degrees of freedom are not very large.. The Mandel-Paule aroach has the advantage of modest comlexty and consstency under normalty assumtons, resultng n a better estmate of the excess varance than the DerSmonan-Lard aroach when all degrees of freedom are large and normalty can be assumed. The dsadvantages nclude a need for teratve soluton, and, lke the DerSmonan-Lard method, lack of treatment of degrees of freedom n the ndvdual uncertantes and the same estmate as the Graybll-Deal mean for aarently consstent data. 3. Maxmum lkelhood estmaton s caable of takng account of fnte degrees of freedom n reorted uncertantes and restrcted maxmum lkelhood estmaton rovdes mnmally based varance estmates. The rncal dsadvantage s comaratve comlexty n mlementaton, resultng n few currently avalable software mlementatons. In ractce, however, all three often roduce very smlar estmates for a gven data set (as s the case for the examles here). An aarent dsadvantage of all excess-varance aroaches s the drastc effect at tmes on degrees of equvalence. Because the same estmate of excess varance s used for all laboratores, laboratores wth smaller reorted uncertantes wll generally be allocated DoE uncertantes that are much larger than ther reorted uncertanty. Ths statement s a consequence of a) the need to ncororate an addtonal varance to rovde sensble estmates of KCRV and ts assocated uncertanty when the reorted uncertantes do not account for the observed dserson, and b) a strct nterretaton of the defnton of the degree of equvalence gven n the MRA. One of the examles here shows DoEs formed wth and wthout ths nterlaboratory varance. 5 Concludng remarks An aroach for the statstcal analyss of CCQM key comarson data that makes allowance for an unexlaned nterlaboratory effect s suggested for rovdng a KCRV, ts assocated standard uncertanty and DoEs. It ales when the measured values rovded by the artcants n the key comarson are mutually ndeendent. Such an aroach offers a reasonable comromse when the data taken as a whole cannot be exlaned by the standard uncertantes rovded by the comarson artcants. The secfc aroach suggested s based on the DerSmonan-Lard mean [9][10]. Imlementaton detals of the aroach are rovded and suortng software can be made avalable. Illustratve examles of CCQM key comarsons are gven. The aroach resonds to the remt of the CCQM KCRV WG for greater harmonzaton n key comarson data analyss. It s suggested that CCQM WGs test ths aroach alongsde ther ongong customzed data analyss. Some members of CCQM refer to treat each key comarson on an ndvdual bass rather than use some rescrtve aroach. Wth adequate access to rofessonal statstcal advce, that atttude s commendable. The ntroducton of a varance relatng to nterlaboratory effects assures consstency of the key comarson data wth the KCRV, but wth current nterretaton of the CIPM MRA has the consequence that all laboratores have DoE uncertantes that guarantee mutual consstency and that may be consderably larger than reorted 5

6 uncertantes. It s not yet clear whether alternatve calculatons of u(d ) are defensble gven the alcable assumtons. Annex A. Ratonale for the choce of estmator When key comarson data are mutually nconsstent, there s a need to force consstency n order to rovde a) a meanngful KCRV and value comonents of the DoEs, and b) credble uncertantes assocated wth these values. A consderable number of relevant aers n the statstcal and metrologcal lterature exst that concentrate on excess-varance estmators for ths urose [3][4][8][9][10][1][13][15]. Also see CCQM []. Weghted mean statstcs (where the locaton estmate s exressed as a lnear combnaton of the x ) are manly used for ths urose. See the revew by Rukhn [11]. Such estmators nclude Mandel and Paule (MP), Vangel and Rukhn (VR), DerSmonan and Lard (DL) and the maxmum-lkelhood estmate (MLE). Rukhn [11] gves results of smulatons that ndcate that MP consstently outerforms MLE and ts varants. Moreover, an MLE soluton s governed by a non-lnear algebrac equaton, whch generally nvolves a non-monotonc functon [13][15], for whch there s a ossblty of a non-unque soluton. MP s also gven by the soluton of a non-lnear algebrac equaton. However, the functon nvolved s monotonc and convex [6], wth the result that a unque soluton always exsts. Ths soluton can straghtforwardly be determned by an algorthm wth guaranteed convergence [8][15]. Aroxmatons to MP exst [3][4][8][11][1][13], wth several, artcularly DL [4], beng effectve. Rukhn s smulatons [11] demonstrate that DL erforms almost as well as MP. DL has the advantage that t s a drect method, requrng mlementaton of only a small number of formulae. There could concevably be cases, not covered by Rukhn s smulatons [11], where DL does not erform so well. As mentoned, software for an mlementaton of DL can be made avalable. Annex C. Illustratve examles Presented for each of three examles of CCQM key comarsons s (a) a grah of the data (for each laboratory the measured value and ±1 standard uncertanty assocated wth that value, wth a green bar f used n obtanng the KCRV and red otherwse), the KCRV gven by the MP estmate (black horzontal lne) and ±1 standard uncertantes (blue horzontal lnes) assocated wth that estmate, (b) a table gvng the MLE, MP and DL estmates and the assocated standard uncertantes, and the excess standard uncertanty n each case, and (c) a grah of the DoEs. The thrd examle show the DoEs comuted wth and wthout the estmated nterlaboratory varance. 6

7 K61 Plasmd DNA n soluton (fg μl 1 ) Fgure 1. Laboratory data and KCRV (DL) for K61 Plasmd DNA n soluton Table. MLE, MP and DL estmates, standard uncertantes and excess standard uncertanty for K61 Estmator Estmate Std unc Excess std unc MLE MP DL Fgure. DoEs for K61. 7

8 K5 '-DDE n fsh ol (μg/g) Fgure 3. Laboratory data and KCRV (DL) for K5 '-DDE n fsh ol Table 3. MLE, MP and DL estmates, standard uncertantes and excess standard uncertanty for K5 Estmator Estmate Std unc Excess std unc MLE MP DL Fgure 4. DoEs for K5. 8

9 K5 PCB170 n Sedment (ng g 1 ) Fgure 5. Laboratory data and KCRV (DL) for K5 PCB170 n Sedment Table 4. MLE, MP and DL estmates, standard uncertantes and excess standard uncertanty for K5 PCB170 n Sedment Estmator Estmate Std unc Excess std unc MLE MP DL Fgure 6. DoEs for K5 PCB170 n Sedment, ncludng nterlaboratory varance. 9

10 Fgure 7. DoEs for K5 PCB170 n Sedment, excludng nterlaboratory varance. References [1] CCQM Data evaluaton rncles for CCQM key comarsons. CCQM, 009. [] CCQM CCQM gudance note: estmaton of a consensus KCRV and assocated degrees of equvalence. CCQM, 010. [3] R. DerSmonan and R. Kacker, Random-effects model for meta-analyss of clncal trals: An udate, Contemorary Clncal Trals, 007, 8, [4] R. DerSmonan and N. Lard, Meta-analyss n clncal trals, Controlled Clncal Trals, 1986, 7, [5] F. A. Graybll and R. B. Deal, Combnng unbased estmates. Bometrcs, 1959, 15, [6] H. K. Iyer, C. M. J. Wang and T. Mathew, Models and confdence ntervals for true values n nterlaboratory trals, J. Amer. Statst. Assoc., 004, 99, [7] JCGM 100:008. Gude to the exresson of uncertanty n measurement (GUM). [8] R. N. Kacker, Combnng nformaton from nterlaboratory evaluatons usng a random effects model, Metrologa, 004, 41, [9] J. Mandel and R. Paule, Interlaboratory evaluaton of a materal wth unequal number of relcates, Anal. Chemstry, 1970, 4, [10] R. Paule and J. Mandel, Consensus values and weghtng Factors, J. Research Natl. Bureau Standards, 198, 87, [11] A. L. Rukhn, Weghted means statstcs n nterlaboratory studes, Metrologa, 009, 46, [1] A. L. Rukhn and N. Sedransk, Statstcs n metrology: nternatonal key comarsons and nterlaboratory studes, J. Data Scences, 007, 5, [13] L. Rukhn and M. G. Vangel, Estmaton of a common mean and weghted means statstcs, J. Amer. Statst. Assoc., 1998, 93, [14] K. Wese and W. Wöger, Removng model and data non-conformty n measurement evaluaton, Meas. Sc. Technol., 000, 11, [15] R. Wllnk, Statstcal determnaton of a comarson reference value usng hdden errors, Metrologa, 00, 39,