Error analysis in biology

Size: px

Start display at page:

Download "Error analysis in biology"

Jasper Lloyd
5 years ago
Views:

1 Error analysis in biology Marek Gierliński Division of Computational Biology Hand-outs available at

2 Oxford Latin dictionary 2

3 Previously on Errors Random variable: numerical outcome of an experiment Probability distribution: how random values are distributed Discrete and continuous probability distributions Gaussian (normal) distribution very common 95% probability within μ ± 1.96σ Poisson (count) distribution random and independent events mean = variance approximates Gaussian for large n Binomial distribution probability of k successes out of n trials toss a coin approximates Gaussian for large n 3

4 Example Take one cuvette with bacterial culture Measure optical density (OD600) Result: 0.37 Reading error Take five cuvettes and find mean OD600 Results 0.42 Sampling error These are examples of measurement errors 4

5 2. Measurement errors If your experiment needs statistics, you ought to have done a better experiment Ernest Rutherford

6 Systematic and random errors Systematic errors your mistakes Random errors statistics sucks Incorrect instrument calibration Change in experimental conditions Pipetting error Reading errors Sampling errors Counting errors Intrinsic variability 6

7 YOU NEED REPLICATES

8 Reading error The reading error is half of the smallest division Example: mm from a ruler smallest division Beware of digital instruments that sometimes give readings much better than their real accuracy Read the instruction manual! Reading error does not take into account biological variability 8

used at this point accuracy of NaOH concentration weight of solid NaOH dissolved volume of water added measured value uncertain

9 Random measurement error Determine the strength of oxalic acid in a sample Method: sodium hydroxide titration Uncertainties contributing to the final result volume of the acid sample judgement at which point acid is neutralized volume of NaOH solution used at this point accuracy of NaOH concentration weight of solid NaOH dissolved volume of water added measured value uncertain result contributing errors Each of these uncertainties adds a random error to the final result Measurement errors are normally distributed 9

10 Random measurement error data Gaussian Gene expression from RNA-seq in 42 replicates 10

11 Counting error Dilution plating of bacteria Found C = 10 colonies 10 Counting statistics: Poisson distribution σ = μ Use standard deviation as error estimate S = C = 10 3 C = 10 ± 3 11

Counting error Gedankenexperiment Measure counts on 10,000 plates True unknown mean C i Count from plate i C i S i = C i Its error μ Unknown population

12 Counting error Gedankenexperiment Measure counts on 10,000 plates True unknown mean C i Count from plate i C i S i = C i Its error μ Unknown population mean S i σ = μ Unknown population SD Counting errors, S i, are similar, but not identical, to σ C i is an estimator of μ S i is an estimator of σ μ ± σ 12

13 Exercise: is Dundee a murder capital of Scotland? On 2 October 2013 The Courier published an article Dundee is murder capital of Scotland Data in the article (2012/2013): City Murders Per 100,000 Dundee Glasgow Aberdeen Edinburgh Compare Dundee and Glasgow Find errors on murder rates Hint: find errors on murder count first 13

14 Exercise: is Dundee a murder capital of Scotland? City Murders Per 100,000 Dundee Glasgow ΔC D = ΔC G = Errors scale with variables, so we can use fractional errors ΔC D C D = 0.41 ΔC G D G = 0.23 and apply them to murder rate ΔR D = = 1.7 ΔR G = = 0.74 p =

15 Exercise: is Dundee a murder capital of Scotland? City Murders Per 100,000 Dundee Glasgow Aberdeen Edinburgh % confidence intervals (Lecture 4) p-values from chi-square test vs Dundee p = 0.8 p = 0.04 p =

17 Sampling error Repeated measurements give us mean value variability scale Sampling from a population Measure the weight of a potato Sample: 5 potatoes Population: all potatoes Small sample size introduces uncertainty Body weight of 5 potatoes (g) Mean (g)

18 Measurement errors: summary Random measurement errors are expected to be normally distributed Some errors can be estimated directly reading (scale, gauge, digital read-out) counting Other uncertainties require replicates (a sample) this introduces sampling error 18

19 Example Weight of 5 potatoes This is a sample We can find mean = 150 g median = 150 g standard deviation = 26 g standard error = 12 g These are examples of statistical estimators No. Weight (g)

20 3. Statistical estimators The average human has one breast and one testicle Des MacHale

21 Population and sample Sample selection Terms nicked from social sciences Most biological experiments involve sample selection Terms population and sample are not always literal 21

22 What is a sample? The term sample has different meanings in biology and statistics biological samples (specimens) Biology: sample is a specimen, e.g., a cell culture you want to analyse quantification Statistics: sample is (usually) a set of numbers (measurements) In these talks: x 1, x 2,, x n Statistical sample (set of numbers)

23 Population and sample Population unknown parameters μ, σ, Sample size n known statistics M, SD, A parameter describes a population A statistical estimator (statistic) describes a sample A statistical estimator approximates the corresponding parameter 23

24 Sample size 10 Dilution plating experiment What is the sample size? n = 1 This sample consists of one measurement: x 1 = colonies 24

25 What is a statistical estimator? Right and lawful rood * from Geometrei, by Jacob Köbel (Frankfurt 1575) *rood a unit of measure equal to 16 feet Stand at the door of a church on a Sunday and bid 16 men to stop, tall ones and small ones, as they happen to pass out when the service is finished; then make them put their left feet one behind the other, and the length thus obtained shall be a right and lawful rood to measure and survey the land with, and the 16th part of it shall be the right and lawful foot. Over 400 years ago Köbel: introduced random sampling from a population required a representative sample defined standardized units of measure used 16 replicates to minimize random error calculated an estimator: the sample mean 25

Statistical estimators Statistical estimator is a sample attribute used to estimate a population parameter μ M σ SD From a sample x 1, x 2,, x n we can find population N(20,

26 Statistical estimators Statistical estimator is a sample attribute used to estimate a population parameter μ M σ SD From a sample x 1, x 2,, x n we can find population N(20, 5) n M = 1 n i=1 x i mean sample n = 30 SD = 1 n n 1 i=1 x i M 2 standard deviation median, proportion, correlation, n = 30 M = 20.3 g SD = 5.2 g SE = 0.94 g M = 20.3 ± 0.9 g 26

27 Standard deviation Standard deviation is a measure of spread of data points Idea: calculate the mean find deviations from the mean of individual points get rid of negative signs combine them together Sample mean Deviation from the mean 27

28 Standard deviation Standard deviation is a measure of spread of data points Idea: calculate the mean find deviations from the mean of individual points get rid of negative signs combine them together Sample mean Deviation from the mean Standard deviation of x 1, x 2,, x n SD n = 1 n i x i M 2 SD n 1 = 1 n 1 i x i M SD n 1 estimates true variance better than SD n Mean deviation MD = 1 n i x i M doesn t overestimate outliers less accurate than SD mathematically more complicated tradition: use SD 28

29 Sampling distribution Population of mice with Gaussian body weight: μ = 20 g, σ = 5 g Draw lots of samples of size n = 5 Population Sample means Sample mean weight (g) 29

30 Normalized frequency Sample no. Standard error of the mean Hypothetical experiment 10,000 samples of 5 mice Build a distribution of sample means Width of this distribution is the true uncertainty of the mean σ m = σ n = 2.2 g Real experiment 5 mice Measure body mass: 7.9, 15.3, 18.5, 22.4, 25.3 g Find standard error SE = SD n = 3.0 g Distribution of sample means SE is an approximation of σ m 30

31 Normalized frequency Sample no. Standard error of the mean Hypothetical experiment 10,000 samples of 30 mice Build a distribution of sample means Width of this distribution is the true uncertainty of the mean σ m = σ n = 0.9 g Real experiment 30 mice Measure body mass: 11.6, 13.7,, 32.8 g Find standard error SE = SD n = 0.8 g Distribution of sample means SE is an approximation of σ m 31

32 Standard error of the mean 32

33 Standard deviation and standard error Standard deviation Standard error SD = 1 n 1 i x i M 2 SE = SD n Measure of dispersion in the sample Error of the mean Estimates the true standard deviation in the population, Does not depend on sample size Estimates the width (standard deviation) of the distribution of the sample means Gets smaller with increasing sample size 33

34 Correlation coefficient Two samples: x 1, x 2,, x n and y 1, y 2,, y n r = 1 n n 1 i=1 x i M x SD x y i M y = 1 n Z SD y n 1 xi Z yi i=1 where Z is a Z-score Correlation does not mean causation! 34

35 Correlation coefficient: example x y Z x Z y Z x Z y r = 1 n Z n 1 xi Z yi i=1 Z x Z y = 4.96 x y Z x Z y Z x Z y Z x Z y =

36 Statistical estimators Central point Mean Geometric mean Harmonic mean Median Mode Trimmed mean Dispersion Variance Standard deviation Mean deviation Range Interquartile range Mean difference Skewness Kurtosis Symmetry Dependence Pearson s correlation Rank correlation Distance 36

37 Hand-outs available at Please leave your feedback forms on the table by the door

Error analysis in biology

Error analysis in biology Marek Gierliński Division of Computational Biology Hand-outs available at http://is.gd/statlec Errors, like straws, upon the surface flow; He who would search for pearls must