AE160 Introduction to Experimental Methods in Aerospace Uncertainty Analysis C.V. Di Leo (Adapted from slides by J.M. Seitzman, J.J. Rimoli) 1
Accuracy and Precision Accuracy is defined as the difference between the measurement and the true value. - Something is repeatably wrong with the measurement Precision is defined by the difference in measured values during repeated measurements of the same quantity. - Denotes errors that change randomly each time we repeat the measurement
Some Systematic Measurement Errors u Actual Model u(x=const) Actual Model Decrease in accuracy u x Nonzero offset - Background Model Nonlinearity u time Drift (e.g., offset changing in systematic way with time) Model Quantization Error (digitized data impacts resolution) u backlash Model Actual Actual Actual x x Systematic errors result in decreased accuracy. hysteresis x Backlash & Hysteresis Systematic errors can be eliminated/removed if they are known 3
Some Random Measurement Errors u(x=const) u Actual Model Decrease in precision time Background Noise (offset changing randomly with time) Detector Noise (random change in sensitivity of device) x Random errors decrease the precision of the measurement After the data is acquired, nearly impossible to separate random error (noise) sources We examine random error with statistical methods 4
Uncertainty Analysis We do not know the exact error o If we did, we would correct it and be error free We must estimate the error or uncertainty in our measurement o This requires application of some basic probability and statistics 5
Basic Concepts Assume that n measurements have been taken of x The mean is given as The sample variance is given as s x = n i=1 (x i x) x = n i=1 The mean and standard deviation do not completely describe the data x i n n 1, with s x the standard deviation 6
.., AE 610 - UNCERTAINTY ANALYSIS ASSUME THAT N MEASUREMENTS HAVE BEEN TAKEN OF X. " SAMPLE # 1 n THE MEAN IS N I - { is, I ~ SAMPLE VARIANCE & STANDARD DEVIATION v = 5 = { N -5 - N -1 it Vx - SAMPLE VARIANCE - S = # STANDARD DEVIATION THE MEAN AND STANDARD DEVIATION DO NOT COMPLETELY DESCRIBE THE DATA
Uncertainty Analysis Consider two signals x and y with identical mean and standard deviation 1.5 Amplitude x i 1 0.5 0 0.5 1.5 0 00 400 600 800 1 10 3 i x =0.5 S x =0.3 Amplitude y i 1 0.5 0 0.5 0 00 400 600 800 1 10 3 i Figure 1: Two signals, x and y that each comprise 1000 values and have identical mean Clearly the two signals are different. Lets look at their probability distribution. (fig: H/Hard Lab) 7
. '. ' -.. -. PROBABILITY DISTRIBUTION WHEN WE MAKE REPEATED MEASUREMENTS OF A SYSTEM WE GET A DISTRIBUTION OF RESULTS. f # OF TIMES THAT X IS IN THIS RANGE,... / ( FREQ. / ( ( / ( \, ', \ \. \ I a, I X XEXACT XAVG SYSTEMATIC ( RANDOM UNCERTAINTY UNCERTAINTY
- THE PROBABILITY DISTRIBUTION RELATES TO THE RANDOM UNCERTAINTY CALCULATION OF THE UNCERTAINTY FROM THE STANDARD DEVIATION DEPENDS ON THE NATURE OF THE DISTRIBUTION OF THE UNDERLYING SIGNAL ASSUME THAT THE SIGNAL OBEYS A GAUSSIAN ( Normal ) PROBABILITY DENSITY FUNCTION Pocx,µo)=o reexpetzfot)] µ 0 - STANDARD MEAN DEVIATION
Probability Distributions Consider two signals x and y with identical mean and standard deviation Probability Density 1.5 1 0.5 signal x signal y 0 0.5 0 0.5 1 1.5 Amplitude Signal y obeys a Gaussian probability density function, given by P Gauss (x, µ, )= 1 exp 1 x µ (fig: H/Hard Lab) 9
Probability Distributions Consider two signals x and y with identical mean and standard deviation Probability Density 1.5 1 0.5 signal y Gaussian 0 0.5 0 0.5 1 1.5 Amplitude Signal y obeys a Gaussian probability density function, given by P Gauss (x, µ, )= 1 exp 1 x µ (fig: H/Hard Lab) 10
PROBABILITY RANGE THE PROBABILITY THAT THE SIGNAL 15 BETWEEN TWO VALUES ) X, AND Xz ) IS GIVEN By THE INTEGRAL OF THE PDF Let x. =µ+j,xz=µ - J Coco,µo)= # ftp.esept.tzfx#5fdx ( G IS KNOWN AS A CONFIDENCE INTEGRAL. WE USUALLY DEFINE THE RANGE J in TERMS OF THE STANDARD DEVIATION O LET J= 1. on Co.IE.mP[±zfIt)fdx ( ga 0.68 For oil.o " WE HAVE 68% CONFIDENCE THAT A NEW MEASUREMENT WILL FALL WITHIN II. O OF I "
Probability Range C Gauss (,µ, )= µ µ+ 1 exp 1 x µ dx Example: We can state with 68% confidence that a new measurement will fall within one sigma of the previously measured mean 13
- Is -. CONFIDENCE ON I ± F WE TAKE N ( LARGE ) MEARUMENTS OF I, HOW CONFIDENT CAN WE BE THAT I 15 CLOSE TO THE TRUE MEAN µ? Apply CONFIDENCE INTEGRALS I=µ±ac of Ac FACTOR WHICH DEPENDS ON THE LEUEZ OF CONFIDENCE Of THE STANDARD DEUIATON OF THE MEAN of=± A * MULTIPLE MEASUREMENTS PROVIDE INCREASED EXPERIMENTAL PRECISION! i,
Confidence Levels For Gaussian (Normal) distribution Error Level Name Error Level Prob. that Error is Smaller Prob. that Error is Larger Probable Error ± 0.67 s 50% 1: One Sigma ± s 68% ~ 1:3 90% error ± 1.65 s 90% 1:10 Two Sigma ± 1.96 s 95% 1:0 Three Sigma ± 3 s 99.7% 1:370 Maximum Error ± 3.9 s 99.9 1:1000 Four Sigma ± 4 s 99.994% 1:16000 Six Sigma ± 6 s 99.9999999% 1:1.01e9 Example: - There is a 95% probability that lies in an area defined by x x = µ ± x = µ ± S x N 15
Uncertainty Estimates: Examples Example 1: Find 95% confidence limits for x = 75 psi when S x =8.3 psi for n = 50 samples. Answer: x ± x = 75 ± 8.3 50 psi = 75 ±.3 psi Example : Find 99.9% confidence limits for the previous case Answer: x ± 3.9 x = 75 ± 3.9 8.3 psi = 75 ± 3.9 psi 50 Example 3: How many samples (N) are required to assume that the mean is within ±5% of its true value with 95% confidence? Answer: 75 ± 8.3 N psi = 75 ± (75 0.05) N = 19.6 0 samples 16
Uncertainty Estimates: Examples You have a differential pressure transducer you are using to measure pressure in a wind tunnel. The manufacturer says that the transducer is linear to within 0.05% of full-scale (10 Torr). You measure the pressure 500 times and find the average is 1.5 Torr and the rms is 0.05 Torr. Question: What are the systematic and random uncertainties of the measurement? o Systematic o u systematic = 0.05% of 10 Torr = 0.005 Torr o Random o S x /sqrt(n) = 0.05Torr/Sqrt(500) = 0.001 Torr o Using a 95% Confidence level o u random = x 0.001 Torr = 0.00 Torr 17
Combining Bias and Precision Uncertainties We noted earlier that errors in each measured variable will include both systematic and random components These can usually be treated as independent and therefore the uncertainties for each can be combined into a total u total = u systematic + u random 1/ If they are not independent, combining in other ways may be necessary 18
Uncertainty Estimates: Examples You have a differential pressure transducer you are using to measure the q of a wind tunnel. The manufacturer says that the transducer is linear to within 0.05% of full-scale (10 Torr). You measure the q 500 times and find the average is 1.5 Torr and the rms is 0.05 Torr. Question: What are the systematic and random uncertainties of the measurement? o Systematic o u systematic = 0.05% of 10 Torr = 0.005 Torr o Random o S x /sqrt(n) = 0.05Torr/Sqrt(500) = 0.001 Torr o Using a 95% Confidence level o u random = x 0.001 Torr = 0.00 Torr o Total o u total = (0.005 + 0.00 ) 1/ Torr = 0.0054 Torr We would report a mean pressure of 1.5 ± 0.0054 Torr with 95% Confidence 19
Propagation of Uncertainty Often experimental results are the result of several independent measurements combined using a theoretical formula How do uncertainties in each variable contribute to the whole? 0
Propagation of Uncertainty We measure mass, velocity of the object at time 1, velocity of the object at time, and the time interval Find the accelerating force based on our measurements, including the uncertainty in our measured force. Mass (kg) Velocity 1 (m/s) Velocity (m/s) Δt (ms) Mean 10.1 31.5 34. 55.1 S. Dev 0.1 0.3 0.33 0.1 # of Samples 10 900 900 900 1
Propagation of Uncertainty The accelerating force can be computed simply through F = m v v 1 t Mass (kg) Velocity 1 (m/s) Velocity (m/s) Δt (ms) Force (N) Mean 10.1 31.5 34. 55.1 495 S. Dev 0.1 0.3 0.33 0.1 # of Samples 10 900 900 900 u w/95% confidence Uncertainty Factor for 95% confidence level u =1.96 x, x = s x n S. Dev # of Samples S. Dev of the mean
Propagation of Uncertainty: Example The accelerating force can be computed simply through F = m v v 1 t Mass (kg) Velocity 1 (m/s) Velocity (m/s) Δt (ms) Force (N) Mean 10.1 31.5 34. 55.1 495 S. Dev 0.1 0.3 0.33 0.1 # of Samples 10 900 900 900 u w/95% confidence Uncertainty Factor for 95% confidence level u mass = 0.063 u v1 = 0.01 u =1.96 x, u v = 0.0 x = s x n u dt = 0.007 S. Dev # of Samples? S. Dev of the mean 3
Analysis of Combined Uncertainties Consider the previous problem, which generically may be written as y = y(x 1,x,...,x n ) where y is the desired measurement and x i quantities we can measure. are the We estimate y by using the mean value of the x i ȳ = y( x 1, x,..., x n ) In general, the average of a measured quantity does not coincide with its exact value x i = x i ± xi hence, the estimated and exact values of the derived quantity are also not the same y =ȳ ± ȳ 4
Analysis of Combined Uncertainties ȳ based on our esti- We want to estimate the value of mates of x i. First we may write y(x 1,x,...,x n )=y( x 1 + dx 1, x + dx,..., x n + dx n ) The differential approximation of y around the point defined by the mean values x i is given by y( x 1 + dx 1, x + dx,..., x n + dx n ) = y( x 1, x,..., x n )+ y x 1 dx 1 + y x dx +... + y x n dx n ȳ dy 5
Analysis of Combined Uncertainties The terms in the previous equation for dy might be positive or negative, this implies dy y y y dx 1 + dx +... + dx n x 1 x x n Thus, for u xi uȳ = y x 1 u x1 << 1 we can estimate + y x u x +... + y x n u xn 1/ 6
Propagation of Uncertainty: Example Mass (kg) Velocity 1 (m/s) Velocity (m/s) Δt (ms) Force (N) Mean 10.1 31.5 34. 55.1 495 S. Dev 0.1 0.3 0.33 0.1 # of Samples 10 900 900 900 u w/95% confidence u mass = 0.063 u v1 = 0.01 u v = 0.0 u dt = 0.007 6.3 u F = F m u mass + F v 1 u v1 + F v u v + F t u dt F = m v v 1 t u F = v v 1 t u mass + m tu v1 + m tu v + m( v v 1 ) ( t) u dt 7
Propagation of Uncertainty: Example Mass (kg) Velocity 1 (m/s) Velocity (m/s) Δt (ms) Force (N) Mean 10.1 31.5 34. 55.1 495 S. Dev 0.1 0.3 0.33 0.1 # of Samples 10 900 900 900 u w/95% confidence u mass = 0.063 u v1 = 0.01 u v = 0.0 u dt = 0.007 6.3 The accelerating force we measure is 495 ± 6.3 N with 95% Confidence. 8