Statistics Workshop Ramsey A. Foty, Ph.D. Department of Surgery UMDNJ-RWJMS

Transcription

1 Statistics Workshop 2012 Ramsey A. Foty, Ph.D. Department of Surgery UMDNJ-RWJMS

2 An unsophisticated forecaster uses statistics as a drunkard uses lamp-postsfor support rather than for illumination Andrew Lang ( ) Scottish poet and novelist. Then there is the man who drowned crossing a stream with an average depth of six-inches W.I.E. Gates German Author Statistics: The only science that enables different experts using the same figures to draw different conclusions. Evan Esar American Humorist

3 Topics Why do we need statistics? Sample vs population. Gaussian/normal distribution. Descriptive Statistics. Measures of location. Mean, Median, Mode. Measures of dispersion. Range, Variance, Standard Deviation. Precision of the mean. Standard Error, Confidence Interval. Outliers. Grubb s test. The null hypothesis. Significance testing. Variability. Comparing two means. T-test Group exercise Comparing 3 or more groups. ANOVA Group Excercise Linear Regression. Power Analysis.

4 Hypothesis Testing Observe Phenomenon Propose Hypothesis Design Study Statistics are an important Part of the study design Collect and Analyze Data Interpret Results Draw Conclusions

5 Why do we need statistics? Variability can obscure important findings. We naturally assume that observed differences are real and not due to natural variability. Variability is the norm. Statistics allow us to draw from the sample, conclusions about the general population.

6 Sample vs Population Taking samples of information can be an efficient way to draw conclusions when the cost of gathering all the data is impractical. If you measure the concentration of factor X in the blood of 10 people, does that accurately reflect the concentration of Factor X of the human race in general? How about from 100, 1000, or 10,000 people? How about if you sampled everyone on the planet?

7 Statistical methods were developed based on a simple model: Assume that an infinitely large population of values exists and that your sample was randomly selected from a large subset of that population. Now, use the rules of probability to make inferences about the general population.

8 The Gaussian Distribution If samples are large enough, the sample distribution will be bellshaped. The Gaussian function describing this shape is defined as follows: ; where m represents the population mean and s the standard deviation.

9 An example of a Gaussian distribution

10 Descriptive Statistics Measures of Location A typical or central value that best describes the data. Measures of Dispersion Describe spread (varia.on) of the data around that central value. Mean Median Mode Range Variance Standard Deviation Standard Error Confidence Interval No single parameter can fully describe distribution of data in the sample. Most statistics software will provide a comprehensive table describing the distribution.

11 Measures of Location: Mean Mean More commonly referred to as the average. It is the sum of the data points divided by the number of data points. M = M = M=76.78 microns = 77 microns Migra&on Assay Cell # Distance travelled (Microns)

12 Measures of Location: Median Median for odd sample size The value which has half the data smaller than that point and half the data larger. For odd numbers, you first rank order then pick the middle number. Therefore the 5 th number in the sequence is the median = 62 microns. Migration assay Cell # Distance traveled (microns)

13 Measures of Location: Median Median for even sample size Unranked Ranked Find the middle two numbers then find the value that lies between them. Add two middle ones together and divide by 2. Median is (7+13)/2=10. The median is less sensitive for extreme scores than the mean and is useful for skewed data

14 Measures of Location: Mode Mode Value of the sample which occurs most frequently. It s a good measure of central tendency. The Mode for this data set is 72 since this is the number with the highest frequency in the data set. Not all data sets have a single mode. It s only useful in very limited situations. Data sets can be bi-modal. Marble Color Frequency Black 6 Brown 2 Blue 34 Purple 72 Pink 71 Green 58 Rainbow 34

15 Boxplots Largest observed value that is not an outlier 75 th percenple Median 25 th percenple Smallest observed value that is not an outlier 12, 13, 5, 8, 9, 20, 16, 14, 14, 6, 9, 12, 12 5, 6, 8, 9, 9, 12, 12,12, 13, 14, 14, 16, 20

16 Boxplots are used to display summary statistics

17 Measures of Location do not provide information on spread or variability of the data

18 Measures of Dispersion Describe the spread or variability within the data. Two distinct samples can have the same mean but completely different levels of variability. Which mean has a higher level of variability? 110 ± 5 or 110 ± 25 Typical measures of dispersion include Range, Variance, and Standard Deviation.

19 Measures of Dispersion: Range Cell # Range Distance traveled (microns) The difference between 1 24 the largest and smallest 2 27 sample values It depends only on 4 49 extreme values and 5 62 provides no information 6 78 about how the remaining 7 80 data is distributed For the cell migration data: Largest distance = 200 microns Smallest distance = 24 microns Range = = 176 microns. NOT a reliable measure of dispersion of the whole data set.

20 Measures of Dispersion: Variance Variance Defined as the average of the square distance of each value from the mean. To calculate variance, it is first necessary to calculate the mean score then measure the amount that each score deviates from the mean. The formula for calculating variance is: S 2 = (X M ) 2 N 1

21 Why Square? Squaring makes them all positive numbers (to eliminate negatives, which will reduce the variance. Makes the bigger differences stand out, (10,000) is a lot bigger than 50 2 (2500).

22 N vs N-1 N N- 1 Size of the population Size of the sample

23 For the cell migration data, the sample variance is: S 2 = ( 28)2 +( 50) 2 +(55) 2 +( 53) 2 +(1) 2 +(3) 2 +( 15) 2 +( 38) 2 +(123) 2 8 NOT a very user-friendly statistic.

24 Measures of Dispersion: Standard Deviation Standard Deviation The most common and useful measure of dispersion. Tells you how tightly each sample is clustered around the mean. When the samples are tightly bunched together, the Gaussian curve is narrow and the standard deviation is small. When the samples are spread apart, the Gaussian curve is flat and the standard deviation is large. The formula to calculate standard deviation is: SD = square root of the variance.

25 For this data set, the mean and standard deviation are: 77 ± 57 microns Conclusion: There s lots of scatter in this data set.

26 But then again. This is a fairly small population (n=9). What if we were to count the migration of 90, or 900, or 9000 cells. Would this give us a better sense of what the average migration distance is? In other words, how can we determine whether our mean is precise?

27 Precision of the Mean Standard Error A measure of how far the sample mean is away from the population mean. For our data set: SEM = SD N = 57 9 = 57 3 =19 SEM gets smaller as sample size increases since the mean of a larger sample is likely to be closer to the population mean. Increasing sample size does not change scatter in the data. SD may increase or decrease. Increasing sample size will, however, predictably reduce the standard error.

28 Should we show standard deviation or standard error? Use Standard Deviation If the scatter is caused by biological variability and you want to show that variability. For example: You aliquot 10 plates each with a different cell line and measure integrin expression of each. Use standard error If the variability is caused by experimental imprecision and you want to show the precision of the calculated mean. For example: You aliquot 10 plates of the same cell line and measure integrin expression of each.

29 Precision of the Mean Confidence Intervals Combines the scatter in any given population with the size of that population. Generates an interval in which the probability that the sample mean reflects the population mean is high. The formula for calculating CI: CI = X ± (SEM x Z) X is the sample mean and Z is the critical value for the normal distribution. For the 95% CI, Z=1.96. For our data set: 95% CI=77 ± (19x1.96)=77 ± 32 CI 95%= This means that there s a 95% chance that the CI you calculated contains the population mean.

30 CI: A Practical Example Data set A Data set B Data set A Data set B Mean SD SEM Low 95% CI High 95% CI Between these two data sets, which mean do you think best reflects the population mean and why?

31 SD/SEM/95% CI error bars SD SEM 95% CI

32 Outliers An observation that is numerically distant from the rest of the data. Can be caused by systematic error, flaw in the theory that generated the data point, or by natural variability.

33 How to deal with outliers? In general, we first quantify the difference between the mean and the outlier, then we divide by the scatter (usually SD). Grubb s test Z = mean value SD For the cell migration data set: The mean is 77 microns. The Sample furthest from the mean Is the 200 micron point and the SE is 19. So: Z = = 2.15

34 What does a Z value of mean? In order to answer this question, we must compare this number to a probability value (P) to answer the following question: If all the values were really sampled from a normal population, what is the chance of randomly obtaining an outlier so far from the other values? To do this, we compare the Z value obtained with a table listing the critical value of Z at the 95% probability level. If the computed Z is larger than the critical value of Z in the table, then the P value is less than 5% and you can delete the outlier.

35 For our data set: Z calc (-2.15) is less than Z Tab (-2.21), so P is greater than 5% and the outlier must be retained.

36 Topics Why do we need statistics? Sample vs population. Gaussian/normal distribution. Descriptive Statistics. Measures of location. Mean, Median, Mode. Measures of dispersion. Range, Variance, Standard Deviation. Precision of the mean. Standard Error, Confidence Interval. Outliers. Grubb s test. The null hypothesis. Significance testing. Variability. Comparing two means. T-test Group exercise Comparing 3 or more groups. ANOVA Group Excercise Linear Regression. Power Analysis.

37 The Null Hypothesis Appears in the form H o : µ 1 = µ 2 Where; H o = null hypothesis µ 1 = mean of population 1 µ 2 = mean of population 2 An alternate form is H o : µ 1 - µ 2 =0 The null hypothesis is presumed true until statistical evidence in the form of a hypothesis test proves otherwise.

38 Statistical Significance When a statistic is significant, it simply means that the statistic is reliable. It does not mean that it is biologically important or interesting. When testing the relationship between two parameters we might be sure that the relationship exists, but is it weak or strong?

39 Strong vs weak relationships r 2 = r 2 =1.000

40 Sources of Variability Random Error Caused by inherently unpredictable fluctuations in the readings of a measurement apparatus or in the experimenter's interpretation of the instrumental reading. Can occur in either direction. Systematic Error Is predictable, and typically constant or proportional to the true value. Systematic errors are caused by imperfect calibration of measurement instruments or imperfect methods of observation. Typically occurs only in 1 direction.

41 Some Examples Random Error Type of Error Example How to Minimize SystemaPc Error You measure the mass of a ring three Pmes using the same balance and get slightly different values: g, g, g The electronic scale you use reads 0.05 g too high for all your mass measurements (because it is improperly tared throughout your experiment). Take more data. Random errors can be evaluated through stapspcal analysis and can be reduced by averaging over a large number of observapons. SystemaPc errors are difficult to detect and cannot be analyzed stapspcally, because all of the data is off in the same direcpon (either too high or too low).

42 Repeatability/Reproducibility Repeatability The variation in measurements taken by a single person or instrument on the same item and under the same conditions. An experiment, if performed by the same person, using the same equipment, reagents, and supplies, must yield the same result. Reproducibility The ability of a test or experiment to be accurately reproduced or replicated by someone else working independently. Cold fusion is an example of an unreproducible experiment.

43 Hypothesis Testing Observe Phenomenon Propose Hypothesis Design Study Statistics are an important Part of the study design Collect and Analyze Data Interpret Results Draw Conclusions

44 Comparing Two Means Are these two means significantly different? Variability can strongly influence whether the means are different. Consider these 3 scenarios: Which of these will likely yield significant differences?

45 Comparing Two Means Student t-test Introduced in 1908 by William Sealy Gosset. Gosset was a chemist working for the Guiness Brewery in Dublin. He devised the t-test as a way to cheaply monitor the quality of Stout. He was forced to use a penname by his employer-he chose to use the name Student. N < 30 Independent data points, except when using a paired t-test. Normal distribution for equal and unequal variance Random sampling Equal sample size. Degrees of freedom important. Most useful when comparing 2 sample means.

46 The Student t-test Given two data sets, each characterized by it s mean, standard deviation, and number of samples, we can determine whether the means are significant by using a t-test. Note below that the difference between the means is the same but The variability is very different.

47 An Example Drop # Sample 1 Sample The null hypothesis states that there is no difference in the means between samples: 1) Calculate means. 2) Calculate SDs. 3) Calculate SEs. 4) Calculate t-value. 5) Compare t calc to t tab. 6) Accept/reject H o.

48 Plot Data Box Plot Bar Graph

49 1) Calculate Mean M 1 = X 1 = N M 2 = X = N 2 10 = 341 = 156

50 2) Calculate SD SD 1 = (x i M 1 ) 2 N 1 = ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 9 = = 1631 = 40 SD 2 = (x i M 2 ) 2 N 1 = ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 + ( ) 2 9 = = 854 = 29

51 3) Calculate SE SE 1 = SD 1 N = = = 12.5 = 13 SE 2 = SD 2 N = = = 9.1 = 9

52 4) Caculate the t-statistic Sample 1 Sample 2 Mean SD SE 13 9 N t = (13) 2 + (9) 2 = = = 11.6 Now we have to compare our t-value to a table of critical t-values to determine whether the sample means differ. But.

53 We first have to determine the degrees of freedom. Describe the number of values in the final calculation of a statistic that are free to vary. For our data set, the degrees of freedom is 2N-2 or 2(10)-2 or 20-2=18.

54 Why 18 degrees of freedom. To calculate SD we must first calculate the mean and then compute the sum of the several squared deviations from that mean. While there are n deviations, only n-1 are actually free to assume any value whatsoever. This is because we used an n value to calculate the mean. Since we have 2 data sets, then df=2n-2=18

55 Did you hear the one about the statistician who was thrown in jail? He now has zero degrees of freedom.

56 5) Compare t calc to t tab for 18 df For the 95% confidence level and a df of 18, t tab = Our t-value was Since t calc > t tab, then we must reject the H o and conclude that the sample means are significantly different.

57 Your Turn

58 One-tailed vs two-tailed t-test One-tailed t-test Two-tailed t-test A one-tailed test will test either if the mean is significantly greater than x or if the mean is significantly less than x, but not both. The one-tailed test provides more power to detect an effect in one direction by not testing the effect in the other direction. A two-tailed test will test both if the mean is significantly greater than x and if the mean significantly less than x. The mean is considered significantly different from x if the test statistic is in the top 2.5% or bottom 2.5% of its probability distribution, resulting in a p-value less than 0.05.

59 Paired vs Unpaired t-test Paired The observed data are from the same subject or from a matched subject and are drawn from a population with a normal distribution. Example: Measuring glucose concentration in diabetic patients before and after insulin injection. Unpaired The observed data are from two independent, random samples from a population with a normal distribution. Example: Measuring glucose concentration of diabetic patients versus nondiabetics.

60 Comparing Three or More Means Why not just do multiple t-tests? If you set the confidence level at 5% and do repeated t-tests, you will eventually reject the null hypothesis when you shouldn t i.e you increase your chance of making a Type I error. Number of Groups Number of Comparisons α=

61 Frog Germ Layer Experiment Germ Later Surface Tensions Mul&ple t- test results for significance Endo vs Meso: p= Yes Endo vs Ecto: p= Yes Endo vs Ecto under: p= Yes Meso vs Ecto: p= No Meso vs Ecto under: p= Yes Ecto vs Ecto under: p= Yes 4 groups, 6 possible comparisons, 26% chance of detecting significant difference When non exists.

62 To compare three or more means we must use Analysis of Variance (ANOVA) In ANOVA we don t actually measured variance. We measure a term called sum of squares. There are 3 sum of squares we need to measure. 1) Total sum of squares. Total scatter around the grand mean. 2) Between-group sum of squares. Total scatter of the group means with respect to the grand mean. 3) Within-group sum of squares. The scatter of the scores.

63 Frog Germ Layer Experiment Germ Layer Surface Tensions Anova/MCT

64 Frog Germ Layer Experiment Germ Layer Surface Tensions Comparison T- test Anova/MCT Endo vs Yes No Meso Endo vs. Ecto Endo vs Ecto under Meso vs Ecto Meso vs Ecto under Ecto vs Ecto under Yes Yes No Yes Yes No Yes No Yes Yes

65 ANOVA The fundamental equation for ANOVA is: From this we can calculate the mean sum of squares by dividing the sum of squares by the degrees of freedom. SS Tot = SS BG + SS WG MS BG = SS BG df BG ; MS WG = SS WG df WG We can then calculate the F statistic: F = MS BG MS WG

66 To calculate sums of squares we first need to calculate two types of means. 1) group means ( X ) 2) the grand mean (X) SS total = ( X X) 2 Sum of squares of each sample (X) minus the grand mean. SS BG = ( X X) 2 # groups Sum of squares of each group mean minus the grand mean, multiplied by the number of groups. SS WG = ( x x) 2 Sum of squares of each sample (X) minus the group mean.

67 df for ANOVA To calculate the MS BG and MS WG, we need to know the Df. To determine the df for these two parameters we need to partition: MS BG = SS BG df BG MS WG = SS WG df WG df of SS BG = n-1 of how many groups there are. Therefore for 3 groups, df=2. df of SS WG = n-1 of all groups. Therefore for 30 samples (10 in each of the 3 groups), df=27. We can then compared the F calc to the F tab to determine whether significant differences exist in the entire data set.

68 Your Turn.

69 One-way versus two-way ANOVA One-Way ANOVA 1 measurement variable and 1 nominal variable. For example, you might measure glycogen content for multiple samples of heart, liver, kidney, lung etc Two-Way ANOVA 1 measurement variable and 2 nominal variables. For example, you might measure a response to three different drugs in both men and women. Drug treatment is one factor and gender is the other.

70 ANOVA only tells us that the smallest and largest means likely differ from each other. But what about other means? In order to test other means, we have to run post hoc multiple comparisons tests.

71 Post hoc tests Are only used if the null hypothesis is rejected. There are many, including Tukey s, Bonferroni s, Schefe s, Dunn s, Newman-Keul s. All test whether any of the group means differ significantly. These tests don t suffer from the same issues as performing multiple t-tests. They all apply different corrections to account for the multiple comparisons. Accordingly, some post hoc tests are more stringent than others.

72 Linear Regression The goal of linear regression is to adjust the values of slope and intercept to find the line that best predicts Y from X.

73 More precisely, the goal is to minimize the sum of the squares of the vertical distances of the points from the line. Note that linear regression does not test whether your data are linear. It assumes that your data are linear, and finds the slope and intercept that make a straight line that best fits your data.

74 r 2, a measure of goodness-of-fit of The value r 2 is a fraction between 0.0 and 1.0, and has no units An r 2 value of 0.0 means that knowing X does not help you predict Y. When r 2 equals 1.0, all points lie exactly on a straight line with no scatter. Knowing X lets you predict Y perfectly. linear regression

75 How is r 2 calculated? The left panel shows the best-fit linear regression line. In this example, the sum of squares of those distances (SSreg) equals The right half of the figure shows the null hypothesis -- a horizontal line through the mean of all the Y values. Goodnessof-fit of this model (SStot) is

76 An Example

77 Power Analysis: How many samples are enough? If sample size is too low, the experiment will lack the precision to provide reliable answers to the questions it is investigating. If sample size is too large, time and resources will be wasted, often for minimal gain.

78 Calculation of power requires 3 pieces of information: 1) A research hypothesis. This will determine how many control and treatment groups are required. 2) The variability of the outcomes measure. Standard Deviation is the best option. 3) An estimate of the clinically (or biologically) relevant difference. A difference between groups that is large enough to be considered important. By convention, this is set at 0.8 SD.

79 An Example We would like to design a study to measure two skin barriers for burn patients. We are interested in pain as the clinical outcome using the Oucher scale (1-5). We know from previous studies that the Oucher scale has a SD of 1.5. What is the sample size to detect 1 unit (D) on the Oucher scale. Here s the equation: n = (σ 1 2 +σ 2 2) (z 1 α / 2 + z 1 β ) 2 D 2 Here, α is the critical value of z at (1.96) and β is power at 80% (0.84).

80 n = ( ) ( ) 2 = 35.3 = What would happen to n if our clinically relevant difference was set at 2 Oucher units. Here: n = ( ) ( ) 2 What would happen to n if our clinically relevant difference was set at 0.5 Oucher units. Here: n = ( ) ( ) = 8.8 = = = 142

81 Another Example You want to measure whether aggregates of invasive cell lines are less cohesive than those generated from noninvasive counterparts. You know that SD for the control group is 3dynes/cm and for the invasive group is 2dynes/cm. You set the α at 0.05 (=1.96) and β at 80% (=0.84) and D at 2 dynes/ cm. How many aggregates from each group would you need? n = ( ) ( ) 2 n = n = 2 2 (9 + 4) (2.80) = = 25.5 = 26 Therefore, we need 26 aggregates in each group to be able to reliably detect a difference of 2 dynes/cm cohesivity between invasive and non-invasive cells.

82 In general, how do variability, detection difference, and power influence n? More variability in the data Less variability in the data Detect small differences between groups Detect large differences between groups Smaller α (0.01) Less power (smaller β) Higher n required Fewer n required Higher n required Fewer n required Higher n required Fewer n required

83