How to Describe Accuracy

Transcription

1 OK, so what s s the speed of dark? When everything is coming your way, you're obviously in the wrong lane MARS 450 Thursday, Feb A) Standard deviation B) Student s t-test - Test of a mean C) Q-test Who laughs last, thinks slowest Random Error: Comparison of Random and Systematic Errors results in a scatter of results centered on the true value for repeated measurements on a single sample. How to Describe Accuracy Systematic Error: results in all measurements exhibiting a definite difference from the true value Random Error Systematic Error! Accuracy is determined from the measurement of a certified reference material (CRM)! Accuracy is described in terms of Error Absolute Error (X!) Relative Error (%) 100*(X!)/! where: X The experimental result! The true result (i.e. CRM value) plot of the number of occurrences or population of each measurement (Gaussian curve)

2 Certified Reference Materials Confidence Intervals How Certain Are You?! Certified Reference Materials are available from national standardizing laboratories National Institute of Standards and Technology (US) National Research Council (Canada)! The CRM is analyzed along with the samples and its concentration is determined as if it were a sample with unknown concentration! Accuracy is then evaluated by comparing the determined value with the certified value from the standardizing laboratory! Confidence intervals are the range of values for which there is confidence, at some level (probability), that they incorporate the true value of the sample! The limits depend on the degree of certainty desired or required The limits may be stipulated by a standard protocol, by regulation, or contract Confidence Interval and Limits Normal Distribution Sampling Theorem! A confidence interval suggests that our observed sample statistic deviates from the fixed parameter (i.e. mean) by some unknown and variable amount of error Statistic Parameter ± error Confidence Interval Confidence limits Sample Statistic There is a probability p that the population parameter falls within the interval! If a variable x is normally distributed with a mean µ and a standard deviation!, then the sampling distribution of the mean (x), based on random samples of size n, will also be normally distributed and have a mean µ and a standard deviation! x given by:! Or: " x s x " s N N

3 Confidence Limits! Confidence Limits take different forms depending on the type of data available. When s " #, the conf. limit for! is given by: Standard Normal Distribution The Z Distribution " The standard normal distribution has mean 0 and standard deviation sigma 1.! Z deviation from the mean in population SD units X ± Z " N Confidence Limits Calculating a Confidence Interval! Confidence Limits take different forms depending on the type of data available. When # is unknown, the CL for! is given by: X ± t s N Determine the Mean (x) Determine the Standard Deviation (s) Determine the degrees of freedom (n -1; for the t-table) Look up a value for t based on how confident you want to be (90%, 95%, etc.) Calculate using appropriate formula. Reporting (e.g.! 12 ± 2.0 ppm) µ X ± ts N t is the value of Student s t statistic from a t-table N is the # of observations s is the (sample) std. dev.

4 Student s t Values Student s t Values - An Example (x) ng/g vs. (x) SRM 93.8 ng/g Are these measurements different? (x) ± 5.6 ng/g (x) SRM 93.8 ± 3.7 ng/g Calculating a Similarity/Difference Let s state (hypothesis) that the two Means ((x) 1 and (x) SRM ) are NOT different: H 0 : µ 1 - µ SRM 0 H 1 : µ 1 - µ SRM " 0 Because we have a relatively small sampling size (n<30), we will be using the standard error based on the sample standard deviation (s x ): Student s t Values - An Example (x) ± 5.6 ng/g (x) SRM 93.8 ± 3.7 ng/g Determine the Standard Error (s x ) s x s N s x s x SRM

5 Calculating a Similarity/Difference Let s state (hypothesis) that the two Means ((x) 1 and (x) SRM ) are NOT different: H 0 : µ 1 - µ SRM 0 Student s t Values t t ( x1 " x 2) " ( µ 1 " µ 2 ) ( n 1 "1)s ( n 2 "1)s 2 # & % ( n 1 + n 2 " 2 $ n 1 n 2 ' ( 83.6 " 93.8) " ( 0) ( 8 "1)( 1.8) 2 + ( 15 "1)( 1.0) " 2 t 17.8 # & % ( $ 15' Sampling Distribution of the Means (x) ± 1.6 ng/g (x) SRM 93.8 ± 1.0 ng/g The Q-test Deciding when to reject data points! Q experimental (Q exp ) #x q x n #/w #x q x n # # (suspect value nearest value) # w spread (largest value smallest value)! Note: includes x q! Q exp is then compared to a tabulated Q value called Q critical (Q crit )! If Q exp > Q crit then the questionable point should be discarded These measurements ARE different!

6 Q-test Table Example Q-test calculation Can a questionable result be dropped from this data set? 82.5, 84.1, 81.7, 81.0, 80.8, 80.6, 78.4, and 86.8 Which value is the questionable value? Hint: it will lie at the extremes Let s test 86.8 Example Q-test calculation 82.5, 84.1, 81.7, 81.0, 80.8, 80.6, 78.4, and 86.8 Q exp #x q - x n #/w #( )#/( ) Examine Q table for n 8 (see previous slide) Evaluate whether Q exp > Q crit Reject with 90% confidence, but not at 95% confidence! The method detection limit (MDL) is defined a the minimum concentration of a substance that can be measured and reported with 99% confidence! Implies a sense of statistical information about the variability around the lowest measurable amount (±( confidence limits) # This limit depends upon the ratio of the magnitude of the analytical signal to the size of the statistical fluctuations in the blank signal.

7 Unless the analytical signal is larger than the blank by some multiple k of the random variation in the blank, it is impossible to detect the analytical signal with certainty. 1) The minimum distinguishable analytical signal, S m : And, the MDL is given by: S m S bl + ks bl The distribution of results from running blank samples is not strictly normally distributed. However, when k 3, the confidence level of detection will be 95% in most cases. S m S bl + ( 3 " s bl ) c m S m " S bl m The case of Hg in water [Hg] (ng/l) area Rep#1 Rep#2 Rep#3 Std Dev The case of Hg in water: MDL #1 [Hg] (ng/l) area Rep#1 Rep#2 Rep#3 Std Dev S m S bl + ( 3 " s bl ) S m ( 3" ) c m " ng/L

8 The case of Hg in water: MDL #2 Rep # Area Observations on Statistics! Don t let statistics bend the truth.! Statistics should clarify and solidify the significance of the data Average Std Dev Std Dev Slope Intercept c m c m ( ks) /m ( 3" ) 0.014ng/L Observations on Statistics! Samuel Clemens There are lies damn lies and statistics.