Error Error & Ucertaity The error is the differece betwee a TRUE value,, ad a MEASURED value, i : E = i There is o error-free measuremet. The sigificace of a measuremet caot be judged uless the associate error has bee reliably estimated. Sice the true value, is geerally ukow, the so is E. More o errors Ucertaity Error is ukow. However, the likely errors ca be ESTIMATED. They are called ucertaities., For multiple measuremets, a mea value (also called omial value), ca be calculated. Hece, the error becomes: E = Sice remais ukow, the E is still ukow. The ucertaity,, is a estimate of E as a possible rage of errors. Δ E e.g., a velocity measuremet ca be reported as: V = 110m / s ± 5m / s ucertaity The ucertaity may be epressed as physical uits or as a %-age, e.g., V = 110m / s ± 4.5% relative ucertaity 1
Accuracy The accuracy is a measure (or a estimate) of the maimum deviatio of measured values, i, from the TRUE value, : accuracy = estimate of ma Agai, sice the TRUE value is ukow, either is the maimum deviatio. The accuracy is oly a estimate of the worst error. Usually epressed as a percetage, e.g. accurate to 5% Note this implies that 95% of the values are withi the iterval i Eample: pressure measuremet Give: A measuremet is claimed to be: P = 50 psi ± 5 psi. Required: What is the accuracy of the pressure probe used for makig this measuremet? Aswer: The relative ucertaity P / P is about: ΔP ± P 5 50 ± 0.1 ± 10% The accuracy may be estimated to be (aroud) 10%. Eample More o Accuracy Give: A pressure sesor is claimed to be accurate to 5%. Required: What will be the ucertaity (i psi) i the measuremet of a pressure of 50 psi? Aswer: The accuracy ( ± relative ucertaity) is 5%, so ΔP ± 5% ΔP ± 0.05 P ± 0.05 50 ± 2.5psi P The ucertaity i P i psi is ±2.5 psi, so the measuremet should be reported as follows: P = 50 psi ± 2.5 psi. The questio: Are the measured values accurate? ca be reformulated as Are the measured values close to the true value? Or Are the measured values ubiased? 2
Precisio More o Precisio The precisio is a measure (or a estimate) of the reproducibility (i.e. repeatability) of repeated measuremets. It is geerally epressed as the deviatio of a readig (measuremet), i, from its mea value, : precisio = estimate of ma The mea is ot the same as the true value, uless the measuremet is completely ubiased. Precisio is a characteristic of our measuremet. I this cotet: accuracy precisio i The questio: Are the measured values precise? ca be reformulated as Are the measured values close to each other? Accurate but NOT precise Precise but NOT accurate Neither accurate or precise Both accurate ad precise Types of errors A systematic error is oe that happes cosistetly a bias or costat differece betwee the measuremet ad the true value Huma compoets of measuremet systems are ofte resposible for systematic errors, e.g., systematic errors are commo i readig of a pressure idicated by a iclied maometer. I theory, these ca be aticipated ad/or measured ad the corrected, eve after the fact. A radom error is just that radom ad ucotrollable! 3
How to reduce systematic errors? Calibratio: Check the measurig istrumet agaist a kow stadard. ivolves compariso with either: a)a primary stadard (give by the Natioal Istitute of stadards ad techology NIST e Natioal Bureau of Stadards) b)a secodary stadard (with higher accuracy that istrumet) c)a kow iput source. How to reduce radom errors? There is NO radom error free measuremets. Hece, the radom errors CANNOT be elimiated. However, the easiest ways to reduce radom errors is to take more measuremets because: ON AVERAGE, radom errors ted to cacel out Multi-Sample Measuremets Ucertaity i multi-sample measuremets Multi-sample measuremets are sigificat umber of data collected from eough eperimets so that the reliability of the results ca be assured by statistics. I other words, a sigificat umber of measuremets of the same quatity (for fied system variables) uder varyig test coditios (i.e. differet samples ad/or differet istrumets) so that the ucertaities ca be reduced by the umber of observatios. 4
Arithmetic Mea ad SD The Gaussia or Normal Distributio If each readig is i ad there are readigs, the the arithmetic mea value is give by: The stadard deviatio is give by: Due to radom errors data is dispersed i a Gaussia or Normal Distributio. σ = = i= 1 i= 1 i ( ) i 1 2 This is the distributio followed by radom errors. It is ofte referred to as the "bell" curve as it looks like the outlie of a bell. The peak of the distributio occurs at the mea of the radom variable, ad the stadard deviatio is a commo measure for how "fat" this bell curve is. The mea ad the stadard deviatio are all the iformatio that is ecessary to completely describe ay ormally-distributed radom variable. More o Probability Distributio Fuctios More o Probability Distributio Fuctios The probability for a readig to fall i the rage ± of the mea is 100 %. If you itegrate uder the curve of the ormal distributio from egative to positive ifiity, the area is 1.0. (i.e. 100 %) σ + σ The probability for a readig to fall i the rage ± _ of the mea is about 68 %. Itegratig over a rage withi ± _ from the mea value, the resultig value is 0.6826. 5
More o Probability Distributio Fuctios More o Probability Distributio Fuctios 2σ + 2σ The probability for a readig to fall i the rage ± 2_ of the mea is about 95 %. Itegratig over a rage withi ± 2_ from the mea value, the resultig value is 0.954. 3σ + 3σ The probability for a readig to fall i the rage ± 3_ of the mea is almost 100 %. Itegratig over a rage withi ± 3_ from the mea value, the resultig value is 0.997. Probability for Gaussia Distributio (this is tabulated i ay statistics book) The Logormal distributio Probability 50% 68.3% 86.6% 95.4% 99.7% ± value of the mea 0.6754 1.5 _ 2_ 3_ Most evirometal eposure data has a logormal distributio The same formulas ad statistical methods that apply to a ormal distributio ca be used if we use the logarithum of the sample data For summary data, coversio formulas also may be used = l( GM ) l S = l( GSD) = mea of l( data) l S = std. dev. of l( data) l l 6
Estimatig Ucertaity Ofte we represet the ucertaity as a 95% cofidece iterval. I other words, if I state the ucertaity to be with 95% coficece, I am suggestig that I am 95% sure that ay readig i will be withi the rage ± of the mea. The probability of a sample chose at radom of beig withi the rage ± 2_ of the mea is about 95%. 95% Ucertaity twice the stadard deviatio 2_ Propagatio of errors RMS error: (Est of Std Dev or CV) Additio or Subtractio: z = + y or z = y Multiplicatio or Divisio: z = y or z = /y Products of powers: z = m y Propagatio of errors Ma error Additio or Subtractio: z = + y or z = y Multiplicatio or Divisio: z = y or z = /y Products of powers: z = m y Eample Say we collected 50 well water samples cotaiig Xylee, with a average value X, of 40 ± 16 ppb (95% CI). What does that mea? The ± 16 ppb would represet a 95% cofidece iterval. That is, if you radomly select may samples of water from this well you should fid that 95% of the samples meet the stated limit of 40 ± 16 ppb. This does ot mea that you could t get a sample that has a ylee value of 16 ppb, it just meas that it is very ulikely. 7
Eample (cot d) If we assume that variatios i the water samples follow a ormal distributio ad the ylee cocetratio, X, is withi the rage 40 ± 16 ppb (95%CI). What is the stadard deviatio, _? Ucertaity 95% of cofidece iterval 2_ ± 16 ppb ±2σ σ 8ppb Eample (cot d) If you assume that average = 40 ± 16 ppb (95%CI). Estimate the probability of fidigs a sample from this populatio Xylee cocetratio to 16 ppb. E 16 ppb E ], E 3σ ] Sice _ = 8 ppb, a value of 16 ppb is: (40-16)/8 = 3 _ from the average value of 40 Recall that a ±3 _ iterval cotais 99.7 % of values; Therefore: 100 99.7 P( E < 16 ppb) = = 0.15% 2 Data Quality Cotrol ad Cotrol Charts Outliers Cosider a eperimet i which we measure the weight of te idividual idetical blak filter samples: The scale readigs (i grams) are: 2.41, 2.42, 2.43, 2.43, 2.44, 2.44, 2.45, 2.46, 2.47 ad 4.85 The 4.85 g readig seems too high ad likely represets a error i your measuremet, but what if the readigs were 2.50 or 2.51 g? At what poit ca you flag or toss out suspect readigs? 8
Outliers: Chauveet s Criterio Chauveet s criterio Of a group of readigs, a readig may be rejected if the probability of obtaiig that particular readig is less tha 1/2. The probability distributio used is the ormal distributio. It is applied oce to the complete data set, ad readigs meetig the criterio are elimiated. The mea ad stadard deviatio may the be recalculated usig the reduced data set. The values i which are outside of the rage ± Cσ are declared outliers (errors) ad ca be ecluded for the aalysis. Ecludig outliers ca be cotroversial; proceed with cautio! Number of sample 5 10 15 25 50 100 C 1.65 1.96 2.13 2.33 2.57 2.81 Methodology for discardig outlier: 1. After ruig a eperimet, sort the outcomes from lowest to highest value. The suspect outliers will the be at the top ad/or the bottom of your list 2. Calculate the mea value ad the stadard deviatio. 3. Usig Chauveet s criterio, discard outliers. 4. Recalculate the mea value ad the stadard deviatio of smaller sample ad STOP. (Do ot repeat!) Data Quality Cotrol eample 1. The readigs (i grams) were: 2.41, 2.42, 2.43, 2.43, 2.44, 2.44, 2.45, 2.46, 2.47 ad 4.85 2. Calculate m = 2.68 g ad σ = 0. 76 g 3. Apply Chauveet s criterio (for =10, C=1.96): Ay values, m i, outside the rage: m Cσ mi m + Cσ 1.19 g mi 4. 17 g is a outliers ad should be discarded. 4. Clearly the 4.85 value is a outlier. No other poits are. 9
Cotrol Charts Cotrol charts (Shewhart charts) are a useful graphical tool for performig quality cotrol aalysis developed i the 1920s by Dr. Walter A. Shewhart of the Bell Telephoe Labs. Cotrol charts show a graphical display of a quality characteristic that s measured from a sample versus the sample umber (or time). Cotrol charts cotai: a ceter lie that represets the average value of the measured quality characteristic (the i-cotrol state); Two other horizotal lies, called the upper cotrol limit (UCL) ad the lower cotrol limit (LCL) (the out of cotrol state); The cotrol limits are chose so that if the process is i statistical cotrol, early all of the sample poits will fall betwee them. Typically this is 3*Std. Dev. of the measured quatity. As log as the poits plot withi the cotrol limits, the process is assumed to be i cotrol, ad o actio is ecessary. Cotrol Charts - 2 There are may types of charts for sequetial QC data; Cotrol charts for idividual samples or X charts Cotrol charts for sample meas or X charts A cotrol chart for variability, usig the sample rage (R charts) or stadard deviatio (Sigma charts) All charts have the same geeral form: X = sample mea UCL = X LCL = X Where S is the measure of variability uder ormal coditios + A S A S Xi Cotrol Chart Movig Rage Chart of idividual data (Xi) L [Nitrate] 2 1 0-1 -2-3 -4-5 -6-7 -8-9 Eample of X Chart for blaks L[Nitrate] UCL LCL A movig rage average is calculated by takig pairs of data (1,2), (2,3), (3,4),..., (-1,), takig the sum of the absolute value of the differeces betwee them ad dividig by the umber of pairs (oe less tha the umber of pieces of data). This is show mathematically as: 0 10 20 30 40 50 60 70 Blak umber 10
A estimate of the process stadard deviatio is give by: The three sigma cotrol limits become: Plot the ceterlie XBAR, LCL, UCL, ad the process X(i) Note XBAR=mea of data Xi Chart- cotiued Data Aalysis: Outlie 1. Eamie the data for cosistecy: Poits that do ot appear proper should be flagged / elimiated. Check the etire eperimetal procedure if there are too may icosistet data. 2. Plot or check stadards for cosistecy. 3. Perform a statistical aalysis of data where appropriate (& validate assumptios). 4. Estimate the ucertaities i the results. 11