TAYLOR SERIES METHOD (TSM) FOR UNCERTAINTY PROPAGATION 1

Transcription

1 APPEDIX B TAYLOR SERIES METHOD (TSM) FOR UCERTAITY PROPAGATIO 1 In nearly all experiments, the measured values of different variables are combined using a data reduction equation (DRE) to form some desired result. A good example is the experimental determination of drag coefficient of a particular model configuration in a wind tunnel test. Defining drag coefficient as C D = 2F D ρv 2 A (B.1) one can envision that errors in the values of the variables on the right-hand side of Eq. (B.1) will cause errors in the experimental result C D. A more general representation of a data reduction equation is r = r(x 1,X 2,...,X J ) (B.2) where r is the experimental result determined from J measured variables X i. Each of the measured variables contains systematic (bias) errors and random (precision) errors. These errors in the measured values then propagate through the data reduction equation, thereby generating the systematic and random errors in the experimental result, r. Our goal in uncertainty analysis is to determine the effects of these errors, which result in the random and systematic uncertainties in the result. In this appendix, a derivation of the equation describing uncertainty propagation is presented, comparisons with previously used equations and approaches are discussed, and, finally, the approximations leading to the method we recommend for most engineering applications are described. 1 Appendix B is adapted from Ref. 1. Experimentation, Validation, and Uncertainty Analysis for Engineers, Third Edition 2009 John Wiley & Sons, Inc. ISB: H. W. Coleman and W. G. Steele 257

2 258 TAYLOR SERIES METHOD (TSM) FOR UCERTAITY PROPAGATIO B-1 DERIVATIO OF UCERTAITY PROPAGATIO EQUATIO Rather than present the derivation for the case in which the result is a function of many variables, the simpler case in which the result is a function of only two variables is considered first. The expressions for the more general case will then be presented as extensions of the two-variable case. Suppose that the data reduction equation is r = r(x,y) (B.3) where the function is continuous and has continuous derivatives in the domain of interest. The situation is shown schematically in Figure B.1 for the kth set of measurements (x k,y k ) which is used to determine r k.hereβ xk and ɛ xk are the systematic and random errors, respectively, in the k th measurement of x, with a similar convention for the errors in y and in r. Assume that the test instrumentation and/or apparatus is changed for each measurement so that different values of β xk and β yk will occur for each measurement. Therefore, the systematic errors and random errors will be random variables, so x k = x true + β xk + ɛ xk y k = y true + β yk + ɛ yk (B.4) (B.5) b xk b yk x k y k x true m x x k m y y k y true r k = r(x k, y k ) b rk r k r true r k Figure B.1 result. Propagation of systematic errors and random errors into an experimental

3 DERIVATIO OF UCERTAITY PROPAGATIO EQUATIO 259 ow approximate the function r in the DRE using a Taylor series expansion. Expanding to the general point r k from r true gives r k = r true + r x (x k x true ) + r y (y k y true ) + R 2 (B.6) where R 2 is the remainder term and where the partial derivatives are evaluated at (x true,y true ). Since the true values of x and y are unknown, an approximation is always introduced when the derivatives are evaluated at some measured values (x k,y k ). The remainder term has the form [2] R 2 = 1 [ 2 ] r 2! x 2 (x k x true ) r x y (x k x true )(y k y true ) + 2 r y 2 (y k y true ) 2 (B.7) where the partial derivatives are evaluated at (ζ,χ), which is somewhere between (x k,y k )and(x true,y true ). This term is usually assumed to be negligible, so it is useful to consider the conditions under which this assumption might be reasonable. The factors x k x true and y k y true are the total errors in x and y. Ifthe derivatives are of reasonable magnitude and the total errors in x and y are small, then R 2, containing the squares of the errors, will approach zero more quickly than will the first-order terms. Also, if r(x,y) is a linear function, the partial derivatives in Eq. (B.7) are identically zero (as is R 2 ). eglecting R 2, the expansion gives [taking r true to the left-hand side (LHS)] (r k r true ) = r x (x k x true ) + r y (y k y true ) (B.8) This expression relates the total error δ in the kth determination of the result r to the total errors in the measured variables and using the notation can be written as θ x = r x δ rk = θ x (β xk + ɛ xk ) + θ y (β yk + ɛ yk ) (B.9) (B.10) We are interested in obtaining a measure of the distribution of the δ r values for (some large number) determinations of the result r. The variance of the parent distribution is defined by [ ] σδ 2 1 r = lim( ) (δ rk ) 2 (B.11)

4 260 TAYLOR SERIES METHOD (TSM) FOR UCERTAITY PROPAGATIO Substituting (B.10) into (B.11) but deferring taking the limit gives 1 (δ rk ) 2 = θ 2 1 x (β xk ) 2 + θ 2 1 y (β yk ) θ x θ y β xk β yk + θ 2 x + 2θ 2 x θ x θ y 1 (ɛ xk ) 2 + θy 2 1 β xk ɛ xk + 2θy 2 1 (ɛ yk ) θ x θ y ɛ xk ɛ yk β yk ɛ yk β xk ɛ yk + 2θ x θ y 1 β yk ɛ xk (B.12) Taking the limit as approaches infinity and using definitions of variances similar to that in Eq. (B.11), we obtain σ 2 δ r = θ 2 x σ 2 β x + θ 2 y σ 2 β y + 2θ x θ y σ βx β y + θ 2 x σ 2 ɛ x + θ 2 y σ 2 ɛ y + 2θ x θ y σ ɛx ɛ y (B.13) assuming that there are no systematic error/random error correlations so that in Eq. (B.12) the final four terms containing the βɛ products are zero. Since in reality we never know the σ values exactly, we must use estimates of them. Defining u 2 c as an estimate of the variance of the distribution of total errors in the result, b 2 as an estimate of the variance of a systematic error distribution, and s 2 as an estimate of the variance of a random error distribution, we can write u 2 c = θ 2 x b2 x + θ 2 y b2 y + 2θ xθ y b xy + θ 2 x s2 x + θ 2 y s2 y + 2θ xθ y s xy (B.14) In Eq. (B.14), b xy is an estimate of the covariance of the systematic errors in x and the systematic errors in y that is defined exactly by ( ) 1 σ βx β y = lim( ) β xk β yk (B.15) Similarly, s xy is an estimate of the covariance of the random errors in x and y. In keeping with the nomenclature of the ISO guide [3], u c is called the combined standard uncertainty. For the more general case in which the experimental result is determined from Eq. (B.2), u c is given by u 2 c = J 1 θi 2 b2 i + 2 k=i+1 θ i θ k b ik + J 1 θi 2 s2 i + 2 k=i+1 θ i θ k s ik (B.16) where bi 2 is the estimate of the variance of the systematic error distribution of variable X i, and so on. The derivation to this point was presented by the authors in Ref. 4.

5 DERIVATIO OF UCERTAITY PROPAGATIO EQUATIO 261 o assumptions about type(s) of error distributions are made to obtain the preceding equation for u c. To obtain an uncertainty U r (termed the expanded uncertainty in the ISO guide) at some specified confidence level (95%, 99%, etc.), the combined standard uncertainty u c must be multiplied by a coverage factor K, U r = Ku c (B.17) It is in choosing K that assumptions about the type(s) of the error distributions must be made. An argument is presented in the ISO guide that the error distribution of the result r may often be considered Gaussian because of the central limit theorem, even if the error distributions of the X i are not normal. In fact, the same argument can be made for approximate normality of the error distributions of the X i since the errors typically are composed of a combination of errors from a number of elemental sources. If it is assumed that the error distribution of the result r is normal, the value of K for C percent coverage corresponds to the C percent confidence level t value from the t distribution (Appendix A), so that U 2 r = t2 [ + J 1 θi 2 b2 i + 2 J 1 θi 2 s2 i + 2 k=i+1 k=i+1 θ i θ k b ik θ i θ k s ik ] (B.18) The effective number of degrees of freedom ν r for determining the t value is given (approximately) by the Welch Satterthwaite formula as [3] [ J (θ 2 i s2 i + θ 2 i b2 i ) ] 2 ν r = J {[(θ is i ) 4 /ν si ] + [(θ i b i ) 4 /ν bi ]} (B.19) where the ν si are the number of degrees of freedom associated with the s i and the ν bi are the number of degrees of freedom to associate with the b i. If an s i has been determined from i readings of X i taken over an appropriate interval, the number of degrees of freedom is given by ν si = i 1 (B.20) For the number of degrees of freedom ν bi to associate with a nonstatistical estimate of b i, it is suggested in the ISO guide that one might use the approximation ( ) 2 bi (B.21) ν bi 1 2 b i

6 262 TAYLOR SERIES METHOD (TSM) FOR UCERTAITY PROPAGATIO where the quantity in parentheses is the relative uncertainty of b i. For example, if one thought that the estimate of b i was reliable to within ±25%, ν bi 1 2 (0.25) 2 8 (B.22) Consideration of the 95% confidence t values in Table A.2 reveals that the value of t approaches 2.0 (approximately) as the number of degrees of freedom increases. We thus face the somewhat paradoxical situation that as we have more information (ν r increases), we can take t equal to 2.0 and do not have to deal with Eq. (B.19), but for the cases in which we have little information (ν r small), we need to make the more detailed estimates required by Eq. (B.19). In Section B-3 we examine the assumptions required to discard Eq. (B.19) and to simply use t = 2 (for 95% confidence), but first in Section B-2 we compare the derived approach with those published earlier. B-2 COMPARISO WITH PREVIOUS APPROACHES The purpose of this section is to give an historical perspective of the development of uncertainty analysis methodology. B-2.1 Abernethy et al. Approach An approach that was widely used in the 1970s and 1980s was the U RSS, U ADD technique formulated by Abernethy and co-workers [5] and used in Refs. [6] and [7] and other SAE, ISA, JAAF, RC, USAF, ATO, and ISO standards documents [5]. According to Abernethy et al., U RSS = [B 2 r + (ts r) 2 ] 1/2 (B.23) for a 95% confidence estimate and U ADD = B r + ts r (B.24) for a 99% confidence estimate, where B r is given by [ ] 1/2 B r = (θi 2 B2 i ) (B.25) and B r and the B i values are 95% confidence systematic uncertainty (bias limit) estimates, and [ ] 1/2 s r = (θi 2 s2 i ) (B.26)

7 COMPARISO WITH PREVIOUS APPROACHES 263 where t is the 95% confidence t value from the t distribution for ν r degrees of freedom given by ν r = (θ i s i ) 4 J [(θ is i ) 4 /ν si ] (B.27) Consideration of these expressions in the context of the derivation presented in the preceding section shows that they cannot be justified on a rigorous basis. The U ADD approach has always been advanced on the basis of ad hoc arguments and with results from a few Monte Carlo simulations, but (as argued in the ISO guide [3]) for a 99% confidence level, the t value appropriate for 99% confidence should be used as the value of K in Eq. (B.17) to obtain a 99% confidence estimate for an assumed Gaussian distribution. The Abernethy et al. approaches also ignore the possibility of correlated systematic error effects [taken into account in the b ik covariance terms in Eq. (B.18)], although Ref. 6 does consider this effect in one example. B-2.2 Coleman and Steele Approach Coleman and Steele [4], expanding on the ideas advanced by Kline and McClintock [8] and assuming Gaussian error distributions, proposed viewing Eq. (B.14) as a propagation equation for 68% confidence intervals. A 95% coverage estimate of the uncertainty in the result was then proposed as that given by an equation similar to Eq. (B.23), U 2 r = B2 r + P 2 r (B.28) with the systematic uncertainty of the result defined by B 2 r = J 1 θi 2 B2 i + 2 k=i+1 θ i θ k ρ Bik B i B k (B.29) and the random uncertainty (precision limit or precision uncertainty) of the result given by Pr 2 = θi 2 (P J 1 i) θ i θ k ρ Sik P i P k (B.30) k=i+1 where ρ Bik is the correlation coefficient appropriate for the systematic errors in X i and in X k and ρ sik is the correlation coefficient appropriate for the random errors in X i and in X k. The random uncertainty of the variable X i is given by P i = t i s i (B.31) where t i is determined with ν i = i 1 degrees of freedom.

8 264 TAYLOR SERIES METHOD (TSM) FOR UCERTAITY PROPAGATIO Equations (B.29) and (B.30) were viewed as propagation equations for 95% confidence systematic uncertainties and random uncertainties, and thus this approach avoided use of the Welch Satterthwaite formula. Comparison of uncertainty coverages for a range of sample sizes using this approach and the Abernethy et al. approach have been presented [9]. As stated earlier in reference to the Abernethy et al. approach, consideration of these expressions in the context of the derivation presented in the preceding section shows that they cannot be justified on a rigorous basis. Both the approach of Coleman and Steele [4] and the Abernethy et al. U RSS approach [5] (properly modified to account for correlated systematic uncertainty effects) agree with the 95% confidence form of Eq. (B.18) for large sample sizes that is, i (and ν r ) large enough so that t can be taken as 2.0. B-2.3 ISO Guide Approach The ISO guide [3] was published in late 1993 in the name of seven international organizations: the Bureau International des Poids et Mesures (BIPM), the International Electrotechnical Commission (IEC), the International Federation of Clinical Chemistry (IFCC), the International Organization for Standardization (ISO), the International Union of Pure and Applied Chemistry (IUPAC), the International Union of Pure and Applied Physics (IUPAP), and the International Organization of Legal Metrology (OIML). It is now the de facto international standard. One fundamental difference between the approach of the guide and that of Eqs. (B.16), (B.18), and (B.19) is that the guide uses u(x), a standard uncertainty, to represent the quantities b i and s i used in this book. Instead of categorizing uncertainties as either systematic (bias) or random (precision), the u values are divided into type A standard uncertainties and type B standard uncertainties. Type A uncertainties are those evaluated by the statistical analysis of series of observations, while type B uncertainties are those evaluated by means other than the statistical analysis of series of observations. These do not correspond to the categories described by the traditional engineering usage: random (or precision or repeatability) uncertainty and systematic (or bias or fixed) uncertainty. Arguments can, of course, be made for both sets of nomenclature. Type A and type B unambiguously define how an uncertainty estimate was made, whereas systematic and random uncertainties can change from one category to the other in a given experimental program depending on the experimental process used a systematic calibration uncertainty can become a source of scatter (and thus a random uncertainty) if a new calibration is done before each reading in the sample is taken, for example. On the other hand, if one wants an estimate of the expected dispersion of results for a particular experimental approach or process, the systematic/random categorization is useful, particularly when used in the debugging phase of an experiment [10]. Considering the tradition in engineering of the systematic/random (bias/ precision) uncertainty categorization and its usefulness in engineering experimentation as mentioned above, we have chosen to retain that categorization while

9 ADDITIOAL ASSUMPTIOS FOR EGIEERIG APPLICATIOS 265 adopting the mathematical procedures of the ISO guide. This categorization is also used in an AIAA standard [11], in AGARD [12] recommendations, and in the subsequent revisions [13] of Ref. 6. B-2.4 AIAA Standard [11], AGARD [12], and ASI/ASME [13] Approach One of the authors (H.W.C.) was a participant in the orth Atlantic Treaty Organization (ATO) Advisory Group for Aerospace Research and Development (AGARD) Fluid Dynamics Panel Working Group 15 on Quality Assessment for Wind Tunnel Testing and was the principal author of the methodology chapter in the resulting report [12]. This AGARD report, with minor revisions, was issued in 1995 as an AIAA standard [11]. The recommended methodology is that discussed in Section B-2.2, including the additional large sample assumptions discussed in Section B-3, so that t is taken as 2 unless there are other overriding considerations [12]. The other author (W.G.S.) is vice-chair of the ASME Committee PTC 19.1, which is responsible for the ASI/ASME standard on test uncertainty. He was principal author of the revised methodology in the new standard. This revision also takes t as 2 for most engineering applications. B-2.5 IST Approach Taylor and Kuyatt [14] reported guidelines for the implementation of a ational Institute of Standards and Technology (IST) policy, which states: Use expanded uncertainty U to report the results of all IST measurements other than those for which u c has traditionally been employed. To be consistent with current international practice, the value of k to be used at IST for calculating U is, by convention, k = 2. Values of k other than 2 are only to be used for specific applications dictated by established and documented requirements. (The coverage factor k corresponds to the K used here.) The IST approach is thus that in the ISO guide [3], and no confidence level is associated with U when reported by IST even though the coverage factor is specified as 2.0. B-3 ADDITIOAL ASSUMPTIOS FOR EGIEERIG APPLICATIOS In much engineering testing (e.g., as in most wind tunnel tests) it seems that the use of the complex but still approximate equations (B.18) and (B.19) for U r and ν r derivedinsectionb-1wouldbeexcessively and unnecessarily complicated and would tend to give a false sense of the degree of significance of the numbers computed using them [11, 12]. In this section we examine what additional simplifying approximations can reasonably be made for application of uncertainty analysis in most engineering testing.

10 266 TAYLOR SERIES METHOD (TSM) FOR UCERTAITY PROPAGATIO B-3.1 Approximating the Coverage Factor Consider the process of estimating the uncertainty components b i and s i and obtaining U r. The propagation equation and the Welch Satterthwaite formula are approximate, not exact, and the Welch Satterthwaite formula does not include the influence of correlated uncertainties. In addition, unavoidable uncertainties are always present in estimating the systematic standard uncertainties b i and their associated degrees of freedom, ν bi. In fact, the uncertainty associated with an s i calculated from readings of X i can be surprisingly large [3]. As shown in Figure B.2, for samples from a Gaussian parent population with standard deviation σ, 95 out of 100 determinations of s i will scatter within an interval of approximately ± 0.45σ if the s i are determined from = 10 readings and within an interval of approximately ± 0.25σ if the s i are determined from = 30 readings. (A sample with 31 has traditionally been considered a large sample [6].) This effect seems to have received little consideration in the engineering measurement uncertainty literature. As stated in the ISO guide [3, pp.48 49]: This... shows that the standard deviation of a statistically estimated standard deviation is not negligible for practical values of n. One may therefore conclude that type A evaluations of standard uncertainty are not necessarily more reliable than type B evaluations, and that in many practical measurement situations where the number of observations is limited, the components obtained from type B evaluations may be better known than the components obtained from type A evaluations. Considering the 95% confidence t table (Table A.2), one can observe that for ν r 9 the values of t are within about 13% of the large-sample t value Figure B.2 Range of variation of sample standard deviation as a function of number of readings in the sample.

11 ADDITIOAL ASSUMPTIOS FOR EGIEERIG APPLICATIOS 267 of 2. This difference is relatively insignificant compared with the uncertainties inherent in estimating the s i and b i as discussed above. Therefore, for most engineering applications it is recommended that the central limit theorem be applied yielding a Gaussian error distribution for the result and an assumed value of t = 2 for a 95% level of confidence. (This will be called the large-sample assumption.) This assumption eliminates the need for evaluation of ν r using the Welch Satterthwaite formula and thus the need to estimate all of the ν si and the ν bi. Consideration of the Welch Satterthwaite formula [Eq. (B.19)] shows that because of the exponent of 4 in each term, ν r is most influenced by the number of degrees of freedom of the largest of the θ i s i or θ i b i terms. If, for example, θ 3 s 3 is dominant, then ν r ν s3 9for 3 10 [recalling Eq. (B.20)]. If, on the other hand, θ 3 b 3 is dominant, ν r ν b3 9 when the relative uncertainty in b 3 is about 24% or less [recalling Eq. (B.21)]. If there is no single dominant term but there are M different θ i s i and θ i b i that all have the same magnitude and same number of degrees of freedom ν a,then ν r = Mν a (B.32) If M = 3, for example, ν a would only have to be 3 or greater for ν r to be equal to or greater than 9. Therefore, t can often legitimately be taken as 2 for estimating the uncertainty in a result determined from several measured variables even when the numbers of degrees of freedom associated with the measured variables are very small. If the large-sample assumption is made so that t = 2, the 95% confidence expression for U r becomes U 2 r = 22 [ + J 1 θi 2 b2 i + 2 J 1 θi 2 s2 i + 2 k=i+1 k=i+1 θ i θ k b ik θ i θ k s ik ] (B.33) Thus for the large-sample case, we can define the systematic standard uncertainty of the result as b 2 r = J 1 θi 2 b2 i + 2 k=i+1 θ i θ k b ik (B.34) and the random standard uncertainty of the result as s 2 r = J 1 θi 2 s2 i + 2 k=i+1 θ i θ k s ik (B.35)

12 268 TAYLOR SERIES METHOD (TSM) FOR UCERTAITY PROPAGATIO and Eq. (B. 33) can be written as U 2 r = 22 (b 2 r + s2 r ) (B.36) with Eqs. (B.34) and (B.35) viewed as propagation equations for the systematic standard uncertainties and random standard uncertainties, respectively. The application of Eqs. (B.34) to (B.36) is termed detailed uncertainty analysis and is used once the planning phase of an experimental program is completed, as discussed in detail in Chapter 5. If all the covariance terms in Eq. (B.33) are zero, the equation can be written as U 2 r = 22 [ θ 2 i b2 i + θ 2 i s2 i ] = θi 2 22 (bi 2 + s2 i ) (B.37) or, most compactly, Ur 2 = θi 2 U i 2 (B.38) This equation describes the propagation of the overall uncertainties in the measured variables into the overall uncertainty of the result. Application of Eq. (B.38) is termed general uncertainty analysis and is used in the planning phase of an experiment, as described in detail in Chapter 4. REFERECES 1. Coleman, H. W., and Steele, W. G., Engineering Application of Uncertainty Analysis, AIAA Journal, Vol. 33, o. 10, Oct. 1995, pp Hildebrand, F. B., Advanced Calculus for Applications, Prentice-Hall, Upper Saddle River, J, International Organization for Standardization (ISO), Guide to the Expression of Uncertainty in Measurement, ISO, Geneva, Corrected and reprinted, Coleman, H. W., and Steele, W. G., Some Considerations in the Propagation of Systematic and Random Errors into an Experimental Result, Experimental Uncertainty in Fluid Measurements, ASME FED Vol. 58, ASME, ew York, 1987, pp Abernethy, R. B., Benedict, R. P., and Dowdell, R. B., ASME Measurement Uncertainty, Journal of Fluids Engineering, Vol. 107, 1985, pp American ational Standards Institute/American Society of Mechanical Engineers (ASME), Measurement Uncertainty,PTC Part 1,ASME,ew York, American ational Standards Institute/American Society of Mechanical Engineers (ASME), Measurement Uncertainty for Fluid Flow in Closed Conduits, MFC-2M-1983, ASME, ew York, Kline, S. J., and McClintock, F. A., Describing Uncertainties in Single-Sample Experiments, Mechanical Engineering, Vol. 75, 1953, pp. 3 8.

13 REFERECES Steele, W. G., Taylor, R. P., Burrell, R. E., and Coleman, H. W., Use of Previous Experience to Estimate Precision Uncertainty of Small Sample Experiments, AIAA Journal, Vol. 31, o. 10, 1993, pp Coleman, H. W., Hosni, M. H., Taylor, R. P., and Brown, G. B., Using Uncertainty Analysis in the Debugging and Qualification of a Turbulent Heat Transfer Test Facility, Experimental Thermal and Fluid Science, Vol. 4, o. 6, 1991, pp American Institute of Aeronautics and Astronautics (AIAA), Assessment of Wind Tunnel Data Uncertainty, AIAA Standard S , AIAA, ew York, Advisory Group for Aerospace Research and Development (AGARD), Quality Assessment for Wind Tunnel Testing, AGARD-AR-304, AGARD, Brussels, American ational Standards Institute/American Society of Mechanical Engineers (ASME), Test Uncertainty, PTC , ASME, ew York, 2005 (second revision of Ref. 6). 14. Taylor, B.., and Kuyatt, C. E., Guidelines for Evaluating and Expressing the Uncertainty of IST Measurement Results, IST Technical ote 1297, ational Institute of Standards and Technology, Gaithersburg, MD, Sept. 1994, p. 5.