Lecture 10: Non- parametric Comparison of Loca6on. GENOME 560, Spring 2015 Doug Fowler, GS

Transcription

1 Lecture 10: Non- parametric Comparison of Loca6on GENOME 560, Spring 2015 Doug Fowler, GS 1

2 Review What do we mean by nonparametric? What is a desirable localon stalslc for ordinal data? What are NP equivalents of a one- sample t- test? 2

3 Review What do we mean by nonparametric? DescripLve stats or inference methods that don t depend (as much) on the distribulon of the populalon being sampled What is a desirable localon stalslc for ordinal data? What are NP equivalents of a one- sample t- test? 3

4 Review What do we mean by nonparametric? DescripLve stats or inference methods that don t depend (as much) on the distribulon of the populalon being sampled What is a desirable localon stalslc for ordinal data? Median why? What are NP equivalents of a one- sample t- test? 4

5 Review What do we mean by nonparametric? DescripLve stats or inference methods that don t depend (as much) on the distribulon of the populalon being sampled What is a desirable localon stalslc for ordinal data? Median why? What are NP equivalents of a one- sample t- test? Sign test, Wilcoxon signed rank test summary? 5

6 Goals Comparing the medians of two samples using the Wilcoxon Rank Sum test Comparing the medians of many mutually independent samples using the Kruskal- Wallis test 6

7 Wilcoxon Rank Sum Test Used to test whether two samples are likely to be drawn from the same distribulon or different ones 7

8 Wilcoxon Rank Sum Test Used to test whether two samples are likely to be drawn from the same distribulon or different ones IdenLcal to Mann- Whitney U test 8

9 Wilcoxon Rank Sum Test Used to test whether two samples are likely to be drawn from the same distribulon or different ones IdenLcal to Mann- Whitney U test 9

10 Wilcoxon Rank Sum Test Used to test whether two samples are likely to be drawn from the same distribulon or different ones IdenLcal to Mann- Whitney U test Pool N = n x + n y observalons 10

11 Wilcoxon Rank Sum Test Used to test whether two samples are likely to be drawn from the same distribulon or different ones IdenLcal to Mann- Whitney U test Pool N = n x + n y observalons Arrange into an ordered array, preserving labels 11

12 Wilcoxon Rank Sum Test Used to test whether two samples are likely to be drawn from the same distribulon or different ones IdenLcal to Mann- Whitney U test Pool N = n x + n y observalons Arrange into an ordered array, preserving labels Assign ranks to each element of the array from 1 N 12

13 Wilcoxon Rank Sum Test Used to test whether two samples are likely to be drawn from the same distribulon or different ones IdenLcal to Mann- Whitney U test Pool N = n x + n y observalons Arrange into an ordered array, preserving labels Assign ranks to each element of the array from 1 N The test stalslc T x is the sum of the ranks of X 13

14 Wilcoxon Rank Sum Test Used to test whether two samples are likely to be drawn from the same distribulon or different ones IdenLcal to Mann- Whitney U test Pool N = n x + n y observalons Arrange into an ordered array, preserving labels Assign ranks to each element of the array from 1 N The test stalslc T x is the sum of the ranks of X Reject H 0 if T x is very large or very small compared to possible values of T x for n = N 14

15 Wilcoxon Rank Sum Test - Example Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? 15

16 Wilcoxon Rank Sum Test - Example Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? 16

17 Wilcoxon Rank Sum Test - Example Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? 17

18 Wilcoxon Rank Sum Test - Example Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? Observation Sample Rank 2.5 X 3 X 6.2 X 9.1 X 14.3 X 14.7 X 14.1 Y 15.6 Y 16.7 Y 18

19 Wilcoxon Rank Sum Test - Example Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? Observation Sample Rank 2.5 X 1 3 X X X Y X X Y Y 9 19

20 Wilcoxon Rank Sum Test - Example Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? Observation Sample Rank 2.5 X 1 3 X X X Y X X Y Y 9 20

21 DistribuLon of T x When N is Small Consider a case where n x = 2 and n y = 3 We know ranks must be 1, 2, 3, 4, 5 Again, the issue is how to assign these ranks amongst the samples X and Y 21

22 DistribuLon of T x When N is Small Consider a case where n x = 2 and n y = 3 We know ranks must be 1, 2, 3, 4, 5 Again, the issue is how to assign these ranks amongst the samples X and Y There are ways of assigning five ranks to two samples Each way is equally likely under the null hypothesis so each has a probability of 10% 22

23 DistribuLon of T x When N is Small X Ranks Y Ranks Value of T x Probability 1, 2 3, 4, , 3 2, 4, , 4 2, 3, , 3 1, 4, , 4 1, 3, , 5 2, 3, , 5 1, 3, , 4 1, 2, , 5 1, 2, , 5 1, 2, probability Tx 23

24 DistribuLon of T x When N is Small X Ranks Y Ranks Value of T x Probability 1, 2 3, 4, , 3 2, 4, , 4 2, 3, , 3 1, 4, , 4 1, 3, , 5 2, 3, , 5 1, 3, , 4 1, 2, , 5 1, 2, , 5 1, 2, probability Tx 24

25 DistribuLon of T x When N is Large 25

26 DistribuLon of T x When N is Large Will be normally distributed how do we calculate a z value? 26

27 DistribuLon of T x When N is Large Will be normally distributed how do we calculate a z value? Subtract the value of T x we got from the mean of the sampling distribulon of T x and divide by the standard devialon of the sampling distribulon of T x 27

28 DistribuLon of T x When N is Large Will be normally distributed how do we calculate a z value? 28

29 Wilcoxon Rank Sum Test - Example Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? Observation Sample Rank 2.5 X 1 3 X X X Y X X Y Y 9 29

30 Wilcoxon Rank Sum Test - Example Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? Observation Sample Rank 2.5 X 1 3 X X X Y X X Y Y 9 Accept H 0 30

31 DistribuLon of the PopulaLons How would nonequality of the shape of the populalons from which X and Y are drawn affect the test? 31

32 DistribuLon of the PopulaLons How would nonequality of the shape of the populalons from which X and Y are drawn affect the test? Here, we would erroneously infer that the populalons had different medians median 32

33 DistribuLon of the PopulaLons How would nonequality of the shape of the populalons from which X and Y are drawn affect the test? We must make the assumplon that our distribulons have the same shape median 33

34 How Do We Test Whether The Shapes Are Equal? 34

35 How Do We Test Whether The Shapes Are Equal? Simplest way is to use boxplots/histograms to get a sense of whether the distribulons appear to be similar 35

36 How Do We Test Whether The Shapes Are Equal? Simplest way is to use boxplots/histograms to get a sense of whether the distribulons appear to be similar You can use formal tests for dispersion/scale parameters (e.g. Ansari- Bradley) though you have to equalize the localon of the two distribulons first! 36

37 Wilcoxon Rank Sum Test GeneralizaLon of the Wilcoxon Signed Rank test Used to test whether two samples are likely to be drawn from the same distribulon or different ones IdenLcal to Mann- Whitney U test Onen called Mann- Whitney- Wilcoxon test (nolce alphabelcal order of the dead people) AssumpLons: ObservaLons are independent ObservaLons are drawn from a conlnuous distribulons The values drawn are ordered The shapes of the two distribulons are idenlcal 37

38 Frank Wilcoxon Wilcoxon lived from 1892 to He was a polymath, working as an oilman and a tree surgeon before training as a physical chemist, working in plant research and then in process control in industry. In a single paper in 1945 he published both tests that bear his name. 38

39 Goals Comparing the medians of two samples using the Wilcoxon Rank Sum test Comparing the medians of many mutually independent samples using the Kruskal- Wallis test 39

40 Kruskal- Wallis Test GeneralizaLon of the Wilcoxon rank sum test to 3 or more independent random samples Used to test whether the medians of the samples are equal Nonparametric version of the one- way ANOVA 40

41 Kruskal- Wallis Test GeneralizaLon of the Wilcoxon rank sum test to 3 or more independent random samples Used to test whether the medians of the samples are equal Nonparametric version of the one- way ANOVA 41

42 Kruskal- Wallis Test GeneralizaLon of the Wilcoxon rank sum test to 3 or more independent random samples Used to test whether the medians of the samples are equal Nonparametric version of the one- way ANOVA Pool all observalons 42

43 Kruskal- Wallis Test GeneralizaLon of the Wilcoxon rank sum test to 3 or more independent random samples Used to test whether the medians of the samples are equal Nonparametric version of the one- way ANOVA Pool all observalons Rank the pooled samples 43

44 Kruskal- Wallis Test GeneralizaLon of the Wilcoxon rank sum test to 3 or more independent random samples Used to test whether the medians of the samples are equal Nonparametric version of the one- way ANOVA Pool all observalons Rank the pooled samples Sum the ranks for each sample to get individual sample rank sums 44

45 Kruskal- Wallis Test Under the null hypothesis, what should be true about the relalonship between any two rank sums R i, R j? 45

46 Kruskal- Wallis Test Under the null hypothesis, what should be true about the relalonship between any two rank sums R i, R j? Any R i is a random sample of ranks Therefore, the means of any two rank sums should be equal 46

47 Kruskal- Wallis Test Under the null hypothesis, what should be true about the relalonship between any two rank sums R i, R j? Any R i is a random sample of ranks Therefore, the means of any two rank sums should be equal In fact, 47

48 Kruskal- Wallis Test The sum of all the sample rank sums is 48

49 Kruskal- Wallis Test The sum of all the sample rank sums is Where N is the total number of pooled observalons Given that, what is the expected value of any one average rank sum under the null hypothesis (that they are all equal)? 49

50 Kruskal- Wallis Test The sum of all the sample rank sums is Where N is the total number of pooled observalons Given that, what is the expected value of any one average rank sum under the null hypothesis (that they are all equal)? 50

51 Kruskal- Wallis Test StaLsLc Given this, how could we construct a test stalslc to see if each sample median deviates from the expected value? 51

52 Kruskal- Wallis Test StaLsLc Given this, how could we construct a test stalslc to see if each sample median deviates from the expected value? 52

53 Kruskal- Wallis Test StaLsLc Given this, how could we construct a test stalslc to see if each sample median deviates from the expected value? This is the sum of squares (or the sum of the squared differences between each score and the expected value) Variance/standard devialon Least squares regression ANOVA 53

54 Kruskal- Wallis Test StaLsLc Q, the Kruskal- Wallis test stalslc is the weighted sum of squares of devialons of the actual average rank sums from the expected average rank sum 54

55 Kruskal- Wallis Test StaLsLc Q, the Kruskal- Wallis test stalslc is the weighted sum of squares of devialons of the actual average rank sums from the expected average rank sum i th average rank sum value Expected average rank sum value 55

56 Kruskal- Wallis Test StaLsLc Q, the Kruskal- Wallis test stalslc is the weighted sum of squares of devialons of the actual average rank sums from the expected average rank sum Q = 12 P k i=1 n i( R i N+1 2 )2 N(N + 1) 56

57 Kruskal- Wallis Test StaLsLc DistribuLon What distribulon should the KW test stalslc follow (hint, it is not a normal distribulon)? 57

58 Kruskal- Wallis Test StaLsLc DistribuLon What distribulon should the KW test stalslc follow (hint, it is not a normal distribulon)? Another hint: the test stalslc is a sum of squares of something that IS normally distributed 58

59 The Chi- Square DistribuLon The Chi- square distribulon is the distribulon of the sum of squared independent standard normal RVs. df 2 2 χdf = Z ; where Z ~ Ν 0, 1) i= 1 The expected value and variance of the chi- square E(x) = df Var(x) = 2 * (df) 59

60 Kruskal- Wallis Test Example Let s say we survey incoming Genome Sciences classes for the distance each student traveled to get here Year 1 Year 2 Year 3 Year

61 Kruskal- Wallis Test Example Let s say we survey incoming Genome Sciences classes for the distance each student traveled to get here Year 1 Y1 Ranks Year 2 Y2 Ranks Year 3 Y3 Ranks Year 4 Y4 Ranks Rank the observalons colleclvely 61

62 Kruskal- Wallis Test Example Let s say we survey incoming Genome Sciences classes for the distance each student traveled to get here R Year 1 Y1 Ranks Year 2 Y2 Ranks Year 3 Y3 Ranks Year 4 Y4 Ranks sum mean Calculate the rank sum, R, for each class 62

63 Kruskal- Wallis Test Example Let s say we survey incoming Genome Sciences classes for the distance each student traveled to get here R R Year 1 Y1 Ranks Year 2 Y2 Ranks Year 3 Y3 Ranks Year 4 Y4 Ranks sum mean Calculate the rank sum average,, for each class R 63

64 Kruskal- Wallis Test Example Let s say we survey incoming Genome Sciences classes for the distance each student traveled to get here Year 1 Y1 Ranks Year 2 Y2 Ranks Year 3 Y3 Ranks Year 4 Y4 Ranks sum mean Calculate Q Q = 12 P k i=1 n i( R i N+1 2 )2 N(N + 1) 64

65 Kruskal- Wallis Test Example Let s say we survey incoming Genome Sciences classes for the distance each student traveled to get here Year 1 Y1 Ranks Year 2 Y2 Ranks Year 3 Y3 Ranks Year 4 Y4 Ranks sum mean Q = 12 P k i=1 n i( R i N+1 2 )2 N(N + 1) p =

66 Kruskal- Wallis Test Outcome Given the way the test stalslc/hypotheses are constructed, what does a rejeclon of H 0 mean? 66

67 Kruskal- Wallis Test Outcome Given the way the test stalslc/hypotheses are constructed, what does a rejeclon of H 0 mean? That the medians are not all equal (i.e. doesn t tell you which are unequal) 67

68 Kruskal- Wallis Test Outcome Given the way the test stalslc/hypotheses are constructed, what does a rejeclon of H 0 mean? That the medians are not all equal (i.e. doesn t tell you which are unequal) Having rejected the null, you might naturally want to know which medians are different 68

69 Kruskal- Wallis Test Outcome Given the way the test stalslc/hypotheses are constructed, what does a rejeclon of H 0 mean? That the medians are not all equal (i.e. doesn t tell you which are unequal) Having rejected the null, you might naturally want to know which medians are different Pairwise mullple- teslng- corrected Wilcoxon rank sum tests are a way to do this, but you ll have to correct for mullple tests 69

70 Kruskal- Wallis Test GeneralizaLon of the Wilcoxon rank sum test to 3 or more independent random samples Used to test whether the medians of the samples are equal Nonparametric version of the one- way ANOVA AssumpLons: k mutually independent random samples measured on at least an ordinal scale drawn from a conlnuous distribulon shapes of the distribulons are idenlcal 70

71 Nonparametric LocaLon Tests Can be used to perform one or two sample tests with fewer assumplons about the distribulon from which the sample(s) are drawn Usage of sign and rank (rather than interval, as with parametric tests) enable this and confer other benefits More robust (immune to outliers) Can be used on ordinal data NP tests slll have assumplons, and slll must be used with care (e.g. zeroes for sign test, Les, similarity of distribulons for rank- sum test) 71

72 AND THERE IS NO FREE LUNCH Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? Observation Sample Rank 2.5 X 1 3 X X X Y X X Y Y 9 72

73 AND THERE IS NO FREE LUNCH Let s say we have measured a transcript level in 6 pre- operalve palents (X) and 3 post- operalve palents (Y). Does the surgery change transcript levels? If we assume normality and idenlcality of variance, then a two sample t- test gives: 73

74 AND THERE IS NO FREE LUNCH Generally speaking, nonparametric tests trade fewer assumplons for less power 74

75 AND THERE IS NO FREE LUNCH Generally speaking, nonparametric tests trade fewer assumplons for less power Different nonparametric tests perform beser or worse in this regard (efficiency) 75

76 AND THERE IS NO FREE LUNCH Generally speaking, nonparametric tests trade fewer assumplons for less power Different nonparametric tests perform beser or worse in this regard (efficiency) All will do beser than their parametric counterparts when assumplons are violated 76

77 AND THERE IS NO FREE LUNCH Generally speaking, nonparametric tests trade fewer assumplons for less power Different nonparametric tests perform beser or worse in this regard (efficiency) All will do beser than their parametric counterparts when assumplons are violated The Mann- Whitney- Wilcoxon test is parlcularly good, giving up lisle power even for normally distributed data 77

78 R Goals ExecuLng nonparametric tests in R Playing around with different distribulon shapes and test assumplons Examining effect size vs. test outcome 78

79 Reading/Resources hsp:// StaLsLcs/buson/2 hsp://sci2s.ugr.es/keel/pdf/algorithm/arlculo/ wilcoxon1945.pdf hsp:// edu- docs/center- for- translalonal- science- aclviles- documents/berd- 5-6.pdf hsp:// StaLsLcs- IntroducLon- QuanLtaLve- ApplicaLons/dp/ / ref=sr_1_4? s=books&ie=utf8&qid= &sr=1-4&keywords=n onparametric+stalslcs 79