A MONTE CARLO ANALYSIS OF THE EFFECTS OF COVARIANCE ON PROPAGATED UNCERTAINTIES

Transcription

1 A MONTE CARLO ANALYSIS OF THE EFFECTS OF COVARIANCE ON PROPAGATED UNCERTAINTIES Ronald Ainswort Hart Scientific, American Fork UT, USA ABSTRACT Reports of calibration typically provide total combined uncertainties at eac individual calibration point wit no information describing te correlation among tose uncertainties. Tis affects te ability of te user of te report of calibration to make an accurate determination of propagated uncertainty values. Altoug covariance is known to affect propagated uncertainties, in tose cases were tere is insufficient information to evaluate te covariance it is generally taken to be zero even wen te assumption is unlikely to be true. Wen correlation coefficients are allowed to vary from zero, te resulting sape of te propagated uncertainty curve varies depending on te specific combination of correlation coefficient values. Wen combinations of correlation coefficients are cosen at random a wide range of propagated uncertainty curves are obtained. Tis type of Monte Carlo simulation is not intended to recreate te pysical insigt witeld by te provider of te report of calibration owever it does demonstrate te magnitude of te problem faced by te user of te calibration certificate. In particular te paper discusses te impact tat te absence of correlation coefficient information as on te propagation of uncertainty curves for an Au/Pt termocouple calibration. Furtermore tis paper discusses te possibility of a maximum likeliood estimator for propagation of uncertainty curves were information about te degree of correlation among uncertainties is not provided. Te simulations sow tat uncertainty in te degree of correlation between te variances of calibration points may ave a demonstrably significant impact on te overall uncertainty of te calibration. 1. INTRODUCTION Te calibration of a temperature sensor generally requires measuring te sensor at known temperatures and te fitting of measured data to an applicable function used to interpolate between te measured datum points. SPRT calibration data are fit to te functions described by te ITS-90, termocouple calibration data may be fit to functions suc as tose described in [1] and [2], and similarly tere are curves wic apply to PRTs and termistors. Te true values of te calibration data are unknown quantities, owever te range of values in wic te true value is believed to be contained is quantified by a standard deviation-like quantity referred to as a standard uncertainty. Uncertainties are used to describe te state of mind of te metrologist and to reflect te knowledge of te metrologist regarding te dispersion of values represented by a particular measurement. Uncertainties in te calibration data result in uncertainty in te interpolation function fit to te calibration data. Tis effect is called te propagation of uncertainty. Just as tere is a curve wic interpolates between measured values, tere is also a curve wic interpolates between te associated uncertainties of te measured values. Tis curve is defined in part by te variance-covariance matrix of te uncertainties of te measured quantities. Te diagonal elements of tis matrix are te variances (te squares of te standard uncertainties) of eac measured value. Te rest of te matrix is filled in

2 by te covariance of eac uncertainty (i.e. two unique standard uncertainties multiplied by teir associated correlation coefficient.) Te sape of te propagation of uncertainty curve and terefore te uncertainties assigned to te interpolated calibration values depend on our knowledge of te correlation between individual uncertainties. Unfortunately most calibration certificates provide only te k=2 uncertainties wit no information regarding te degree of correlation among tose uncertainties. In tis situation only te diagonal elements of te variance-covariance matrix are known. Tis lack of knowledge implies an additional uncertainty tat deserves evaluation and leads to a dispersion of possible curves to coose from. Terefore it is desirable to coose a propagation of uncertainty curve tat represents an unbiased maximum likeliood estimate of te propagated uncertainty. 2. EXAMPLES OF CORRELATED UNCERTAINTIES Metrologists preparing reports of calibration may ave access to muc useful information for evaluating covariance. Take for example te calibration of an Au/Pt termocouple. Uncertainty components tat may be correlated between calibration points are indicated in Table 1. Table 1: Example of some Type B Termocouple Uncertainties and possible subjective degrees of correlation Uncertainties for Termocouple Calibration at Individual Fixed Points Uncertainty in sealed cell fixed point value (reference cell certification) Uncertainty in fixed point value (non-ideal plateau) Uncertainty in ice bat system Uncertainty in ydrostatic ead correction Uncertainty due to non-ideal immersion profile Uncertainty due to inomogeneity (estimated) Uncertainty due to low termal switc termal EMF Uncertainty due to sensitive DVM long term stability Uncertainty due to sensitive DVM calibration Degree of Correlation Wit Oter Fixed Points Weak Correlation Weak Correlation Strong Correlation Uncorrelated Uncorrelated Strong Correlation Strong Correlation Strong Correlation Weak Correlation It is easy to see tat uncertainties in te ice bat would be correlated if te same ice bat was common to eac point of te calibration and te same can be said for te low termal switc. Uncertainties due to inomogeneity are believed to be correlated between fixed points based on data reported in [3]. However it is difficult to directly observe te covariance in practice. Yet it is possible to make judgments about te degree of correlation qualitatively and a metrologist may be able to put upper and lower bounds on te percentage of variation at one temperature tat is explainable by variation at anoter. Te R-square statistic tells us te percentage of variation in te dependent variable tat is attributable to variation in te independent variable. Te square root of te R-square statistic is te correlation coefficient. If we could estimate for example tat variation in te measured voltage at te Tin point due to te ice bat was 90% explainable by te variation in te measured voltage at te Zinc point due to te ice bat ten we could estimate tat te correlation coefficient would be te square root of 0.90, wic is A similar procedure could be used to put upper and lower bounds on te correlation

3 coefficients. One way of syntesizing tis information into correlation coefficients for te total combined uncertainties would be troug a simulation exercise. A Monte Carlo simulation could be used to incorporate te pysical insigt of te metrologist regarding te individual uncertainty components into an estimate of te correlation coefficients of te total combined uncertainties. An example of a covariance matrix in symbolic form is: (1) Clearly, te metrologist providing te report of calibration as te best information regarding correlated uncertainties and is terefore in te best position to estimate propagated uncertainties for is client. A metod for doing so is provided below. 3. MATHEMATICS OF UNCERTAINTY PROPAGATION Te equation referred to by [4] as te law of propagation of uncertainty is based on a first-order Taylor series expansion of te equation: Y = f X, X,..., X ) (2) ( 1 2 N Y is te observed output and te X N are inputs tat may or may not be observed. Solutions to te first-order Taylor series can be obtained directly, by numerical metods, or by a metod recently described using Lagrange polynomials [5]. However it is also possible using matrix metods to solve for te propagation of uncertainty curves directly witout te use of a first-order Taylor series approximation. Tis turns out to be computationally more efficient and demonstrably equivalent to te oter metods. Furtermore te autor believes tat matrix metods provide more insigt into te relationsip between covariance and te propagation of uncertainty and terefore it is te preferred metod for tis paper. Since te autor is unaware of references to tis particular metod in te literature a brief explanation is provided. If te argument of f ( X 1, X 2,..., X N ) is rewritten as te vector X ten Equation (2) becomes: Y = Xb (3) Now te vector b represents te calibration coefficients and te vector X represents te calibration data. If we want to predict wat te interpolated mean response Yˆ would be to some oter set of inputs X ten we can write te relationsip as follows: Yˆ = X b (4)

4 If Y is a random vector wit expected variance-covariance matrix σ 2 { Y} = E{[ Y E{ Y}][ Y E{ Y}] T } and Equation (4) defines te interpolated response Yˆ for any given set of inputs X, it can easily be sown tat: 2 σ { Yˆ} = E{ X σ { b} } (5) 2 T X 1 were te solution to Equation (3) is b = X Y and leads to te equation T σ { b] = E{ X σ { Y}( X ) }. By combining tis result wit Equation (5) we obtain: T σ { Yˆ} = E{ X X σ { Y}( X X ) } (6) Te propagated uncertainties of te response Ŷ for a given input X are te diagonal elements of te variance-covariance matrix σ 2 { Y ˆ }. In contrast, te diagonal elements of te variance-covariance matrix σ 2 { Y} are values estimated by a metrologist troug te usual process of uncertainty analysis. Tey are simply te squares of te standard uncertainties associated wit te vector Y. Te off-diagonal elements are te covariance and teir estimation requires knowledge about te degree of correlation between te uncertainties along te diagonal of te matrix. Te covariance is defined: were r x i, x ) is te correlation coefficient. ( j 1 σ x, x ) = σ ( x,) σ ( x ) r( x, x ) (7) ( i j i j i j Te solution to te over-determined case is similarly derived by coosing te appropriate solution to Equation (3). Equation (7) is were tis approac normally comes to a alt for te user of a calibration certificate because r ( x i, x j ) usually is unknown. Sometimes it is estimated as aving a value of one. At oter times te metrologist uses is best judgment and ignores correlation. However, frequently te case is tat te user of te report knows te true value is probably somewere between te two extremes but does not know ow to evaluate it. 4. MONTE CARLO ANALYSIS Monte Carlo analysis is a statistical simulation metod wit a wide variety of applications. By representing a problem in terms of probability distribution functions (rater tan differential equations) it is possible to draw out numeric solutions to very difficult problems tat would ave been near impossible to solve wit traditional metods. Recall tat one suggested metod of dealing wit covariance is to treat propagated uncertainty as if covariance was zero. In te case of te Au/Pt termocouple calibration described above tis is unlikely to be te expected value of te covariance. Te question arises ten, Is it likely tat setting covariance to zero will result in te most useful

5 description of propagated uncertainty? Wen no information is provided in te report of calibration about te values or likely distributions of te correlation coefficients ten it seems reasonable to model te dispersion of possible values for te correlation coefficients wit a rectangular distribution. In te first case considered, noting is left off te table. Te range of all possible values for correlation coefficients is by definition -1 to 1. However in te second case it is stated as an opinion tat depending on te facts available about te calibration it migt be possible to narrow te likely range of potential correlation coefficients to between 0 and 1. Grapical results of bot cases are sown in te figures below. Uncertainties appropriate for an Au/Pt termocouple calibration by fixed point were assumed in tis simulation. At Sn, Zn, Al and Ag te k=2 values were taken to be 16.8, 15.1, 17.0 and 19.6 mk respectively. By drawing combinations of correlation coefficients from a random set of data, 18,316 curves were generated in one case and 20,000 curves were generated in anoter representing potential propagated uncertainty curves for te assumed uncertainties. Te first case assumed tat correlation coefficients could vary between -1 and 1 and te second case assumed only positive correlation coefficients. Te correlation coefficient matrices were symmetric wit te diagonal set equal to one. As a normalization condition te matrices were required to be positive definite [7] by te following metod. Eac primary submatrix was tested for a positive determinant to ensure tat all eigenvalues would also be positive. A deviation function of te form 2 f ( t) = a t + b t was cosen and te over-determined form of equation (6) was used to generate te curves. Te resulting output was divided into quartiles. Te environment used to conduct te analysis was a system for statistical computation and grapics called R [8]. Te algoritm used for generating te uniformly distributed random numbers needed for te simulation was te Mersenne Twister [9] wic as a period of RESULTS Figure 1 below displays te results of te Monte Carlo Simulation in grapical form. Te dispersion of values generated was divided into quartiles as indicated by te solid lines in Figure 1. Te median value is indicated by te solid line in te center of te curve and te maximum and minimum values obtained at eac temperature are indicated by te outer solid lines. Additionally tree oter curves are sown. Te solid circles represent te special case were all uncertainties approac 100% correlation between calibration points. Te empty circles represent anoter special case were none of te uncertainties are correlated at any of te fixed points. Finally tere is a tird curve indicated by stars wic was calculated as te average of te two curves above wic ad all correlation coefficients set effectively to one for te correlated case or to zero for te uncorrelated case.

6 Figure1: Te range of propagation of uncertainty curves was determined troug a Monte Carlo investigation utilizing 18,316 simulations. Te simulated curves were produced by coosing covariance matrices tat represented random degrees of correlation between calibration points. Correlation coefficients were uniformly distributed between -1 and 1. Correlation coefficient matrices were symmetric, invertible and accepted if and only if tey were positive definite matrices. For tis simulation an over-determined quadratic deviation function wit two degrees of freedom was cosen. It is interesting to note tat in Figure 1 were correlation coefficients were allowed to vary between -1 and 1 tat te uncorrelated case (r ij =0 wen i j else r ij =1) was a fairly useful predictor of te median of te distribution. On te oter and in Figure 2 te average of te correlated and uncorrelated curves was a better approximation to te curve representing te median of all te simulations and may present te easiest metod of calculating a maximum likeliood estimate curve for te propagation of uncertainty wen correlations are expected to be positive. If tis turns out to be true in oter scenarios ten furter researc sould try to determine weter an averaging tecnique would also be useful wen te upper and lower bound on te correlation coefficients are oter tan 0 and 1.

7 Simulated Range of Propagated Uncertainties _ Quartiles of Randomly Correlated Uncertainty Curves Uncertainty (mk) Correlated Uncertainty Curve Uncorrelated Uncertainty Curve Average of Correlated and Uncorrelated Temperature ( C) Figure2: Te range of propagation of uncertainty curves was determined troug a Monte Carlo investigation utilizing 20,000 simulations. Te simulated curves were produced by coosing covariance matrices tat represented random degrees of correlation between calibration points. Correlation coefficients were uniformly distributed between 0 and 1. Correlation coefficient matrices were symmetric, invertible and accepted if and only if tey were positive definite matrices. For tis simulation an over-determined quadratic deviation function wit two degrees of freedom was cosen. Wen negative correlations are believed to be unlikely it appears tat te propagation of uncertainty curve will be misrepresented by an uncorrelated determination. However if no information is present regarding correlation or if it is believed to be equally likely to be positive as it is to be negative ten no evidence as been presented wic contradicts te validity of assuming covariance is equal to zero. ACKNOWLEDGEMENTS Te autor would like to tank Tom Wiandt and Rick Walker for teir tougtful insigts in reviewing te results presented in tis paper.

8 REFERENCES [1] U.S. Government Printing Office, NIST Monograp 175, Wasington D.C.,1993 [2] Burns G. W., Strouse G. F., Liu B. M., Mangum B. W., Gold versus platinum termocouples: performance data and an ITS-90 based reference function, Temperature Its Measurement and Control in Science and Industry, 6, 1992, pp [3] Jaan F., Ballico M., A Study of te Temperature Dependence of Inomogeneity in Platinum-Based Termocouples, Temperature Its Measurement and Control in Science and Industry, 7, 2003, pp [4] BIPM, IEC, IFCC, ISO, IUPAC,IUPAP,OIML (1995). Guide to te Expression of Uncertainty in Measurement [5] Wite D. R., Te Propagation of Uncertainty Wit Non-Lagrangian Interpolation, Metrologia, ,pp [6] Neter, J.,Kutner, M., Nactseim, C. andwasserman, W., Applied Linear Regression Models, Irwin, Cicago, IL,1990. pp [7] Anton, H., Elementary Linear Algebra, Jon Wiley & Sons, New York, NY, pp [8] Hornik (2004), "Te R FAQ", ISBN , ttp:// [9] L Ecuyer, P., and Hellekalek, P. Random number generators: Selection criteria and testing, Lecture Notes in Statistics, Volume 138 : Random and Quasi-Random Point Sets, edited by P. Hellekalek and G. Larcer, Springer, New York, 1998, pp Address of te Autor: Ainswort Ronald, Hart Scientific, 799 East Uta Valley Drive, American Fork UT 84003, USA, tel , fax , Ron.Ainswort@HartScientific.com